Towards a Theoretical Grounding of Climate Resilience Assessments for Smallholder Farming Systems in Sub-Saharan Africa

Resilience assessments are increasingly used to inform management decisions and development interventions across sub-Saharan Africa (SSA). In light of current and future climate change and variability, there is growing interest in applying such tools and frameworks to assess and strengthen the climate resilience of smallholder farming systems. However, these assessments are often undertaken without explicit consideration of the resilience thinking in which they are grounded. This makes it difficult to understand how the conceptual aspects of resilience are translating into resilience assessment practice. This paper provides an important first step in tackling this gap, by identifying and using key characteristics of resilience thinking to evaluate existing resilience assessment tools and frameworks and drawing insights for assessing the climate resilience of smallholder farming systems. We find that power, politics, and agency, identified as important in the resilience literature, are not fully incorporated within current tools and frameworks. This leads to inadequate consideration of spatial and temporal trade-offs. We propose six recommendations for assessing the climate resilience of smallholder farming systems in SSA in order to enhance the linkages between resilience theory and practice. These are: (1) better integrate vulnerability and resilience; (2) recognize that resilience does not equal development or poverty reduction; (3) recognize the benefits and limitations of adopting flexible, participatory approaches; (4) integrate issues of power into assessment tools; OPEN ACCESS Resources 2015, 4 129 (5) target specific systems; and (6) encourage knowledge sharing, empirical studies, and critical evaluation. Our findings contribute to improved understanding of applications of resilience thinking to enhance natural resource management.


Introduction
Researchers, practitioners, and policy makers employ a range of tools and frameworks to enhance the design, implementation and monitoring of policies and programs to increase resilience.Such tools play a crucial role in enabling strategic choices about funding priorities.They also provide evidence for natural resource management and for monitoring and accountability purposes, regarding where investments can build resilience.Smallholder farming systems represent one arena in which there is increasing interest in developing and applying such tools, largely due to concerns about food security and the impacts of climate change on natural resources.This reflects a broader shift to applying resilience thinking to develop multiple tools, frameworks and methodologies, with a view to assessing and strengthening resilience [1,2].
While resilience is accepted as important and, despite critiques, is generally seen as a positive attribute [3], several gaps remain in moving between resilience thinking and resilience assessment.One particularly pressing challenge is that of scarce empirical evidence that critically evaluates the processes and outcomes from resilience assessments [4].Another challenge is the lack of incorporation of published research on the key aspects of resilience thinking into a widely applicable resilience measurement framework [5].Indeed, research has identified important factors that strengthen resilience of a range of different systems [6], yet this information is often only marginally used among practitioners or in policy domains [7].This paper contributes towards addressing this gap, and aims to provide a more substantive theoretical grounding to the development of future climate resilience assessments for smallholder farming systems.
Scientific awareness and understanding about the potential consequences of climate change and variability have improved significantly since the 1980s, with wide acknowledgement of the potential for climate impacts to affect agricultural production and food security [8,9].Rain-fed agriculture has been repeatedly identified as particularly sensitive to climate changes and variability [10][11][12][13].Sub-Saharan Africa (SSA) is projected to be disproportionately affected by the impacts of future climate change and variability due to a high dependence on rain-fed agriculture for food, income and economic growth [14].At the same time, there are growing concerns that smallholder farmers across SSA, who depend on rain-fed agriculture as their main source of livelihood, may not possess the necessary capacity to cope with and adapt to current and future climate impacts [15,16].One approach to assist smallholder farmers is to increase their social, economic, and ecological resilience.
Farming systems, also referred to as agro-ecosystems, are considered as social-ecological systems (SESs) in the resilience literature, and incorporate both biophysical and human components [17].A farming system comprises multiple diverse, individual, but interacting, individual farms [18].
Understood in this way, a farming system is a unit of analysis above individual farm systems [18].The production and social structures of individual farms are highly diverse, suggesting that the ability of smallholder farmers to respond to climate change and variability will be similarly varied, resulting in different farm-scale climate impacts [19].Strengthening the resilience of individual farms therefore forms part of a broader goal to increase the resilience of smallholder farming systems in the face of both current and future climate change and variability.
Decision makers are increasingly interested in strengthening the climate resilience of smallholder farming systems.Yet it is unclear what kinds of tools and frameworks that focus on climate resilience already exist and the extent to which they reflect current resilience thinking.Furthermore, there is a growing number of resilience assessments and a growing interest in linking resilience theory and practice [20].There is thus potential to draw out insights from how characteristics of current resilience thinking are reflected in other practically-applied resilience assessments such that gaps can be addressed in the ongoing development of climate resilience assessments.Herein, tools are defined as including methodologies and approaches aimed at providing practical assessments or measurements of resilience, and frameworks as collections of concepts or ideas that may have been used or applied to inform the development of practical assessments.
The overall aim of this paper is to analyze the resilience literature alongside existing resilience assessment tools and frameworks with a view to guiding a more comprehensive application of resilience thinking.In particular, this paper makes recommendations for future assessment tools and frameworks that focus on assessing the climate resilience of smallholder farming systems in SSA.The paper's objectives are to: (1) summarize current resilience thinking, setting out the key characteristics relating to resilience in the literature; (2) use a systematic approach to identify existing assessment tools and frameworks from a range of different disciplines and sectors; and (3) evaluate the extent to which tools and frameworks in our sample reflect the key characteristics of resilience thinking identified in objective 1. Key characteristics of resilience that are captured within assessment tools are highlighted and gaps are identified.The paper concludes by drawing out recommendations that reflect the key characteristics of resilience thinking that could better ground the development of future tools that seek to assess the climate resilience of smallholder farming systems in SSA.

Unpacking Resilience
Multiple definitions of resilience exist [21].It is commonly defined as the ability to bounce back after an external shock or stress [22], where the resilience of a system is demonstrated by undergoing disturbance, maintaining system functions and controls, and returning to a stable state [23].Such definitions of resilience have roots in ecology and ecosystem dynamics [24].
Resilience is often presented as an antonym of vulnerability [25], where vulnerability to climate change is defined by the Intergovernmental Panel on Climate Change (IPCC) as the degree to which a system is susceptible to, or unable to cope with, adverse effects of climate change, including climate variability and extremes [26].Empirical studies demonstrate that the relationship between resilience and vulnerability is nevertheless complex [7], as systems can be both resilient and vulnerable at the same time depending on the particular shocks and stresses [27].For example, a farm close to a water body may be vulnerable to flooding yet due to its geographical location be highly resilient to drought.This demonstrates that the type of shock that is experienced matters [28,29].
Resilience has evolved to include the dynamics of social and ecological systems and has been applied as a lens through which we can understand and examine how SESs respond to shocks, stresses or perturbations [30].This highlights the importance of understanding the resilience of a system (or part of a system) in relation to a specific shock, stress or perturbation [25], also referred to as specified resilience [31].However, building specified resilience has limitations.For example, focusing on a specific shock or part of a system can cause the system to lose resilience in other ways or be less resilient to other shocks.It also relies on knowing or being able to predict the nature of a specific shock, which is particularly problematic in the face of increasingly variable, dynamic and uncertain climate conditions [32].
While it is possible to assess resilience after a shock or stress, it can also be assessed before exposure takes place through focus on the "inherent characteristics or qualities of social systems that create the potential for harm" [5] (p.599).Resilience can be considered an intrinsic system property or process [33].Berkes and Seixas [34] refer to this as "general resilience".General resilience does not define a particular part of the system or identify specific shocks [31].Empirical studies of general resilience in SESs demonstrate that the social, economic and ecological context of a system is important [35] and further highlight the centrality of financial, political, and institutional factors [36].Building general resilience is considered highly desirable in coping with and adapting to climate change and variability [20], and many resilience assessment results are used to inform investments in general resilience.
To strengthen the resilience of an individual or system, past exposure to shocks and stresses is considered essential [37].Where exposure has taken place, resilience research demonstrates that historically, individuals and systems have been able to successfully negotiate challenges or adverse events [38].Central to this understanding is the notion that resilience is maintained by disturbing and probing at its boundaries [24,39,40].This implies that all individuals or systems can learn from past exposure, suggesting that processes such as social learning are important [41,42].
Indeed, many authors note that in its recent application, resilience requires flexibility, learning and ability to deal with change [43,44].Here a distinction is drawn between adaptability and transformability as different aspects of resilience [31,45].Adaptability refers to the capacity to deal with change and maintain system functions and structures, whereas transformability is "the capacity to create a fundamentally new system when ecological, economic, or social structures make the existing system untenable" [45] (p.5).Studies demonstrate the importance of including transformability as a characteristic of a resilient system [31,46].In line with current resilience thinking, this paper understands resilience to have three dimensions: persistence, adaptability and transformability [16,31].It proposes that general climate resilience, rather than resilience to specific climate shocks, is a desirable farming system characteristic in the face of changing climatic conditions.It also recognizes that strengthening such climate resilience could contribute to building general resilience.
Resilience thinking recognizes the interconnectedness and interdependence of dynamic and interacting factors that comprise SES [3,44,47].SESs are predisposed to change rather than equilibrium and are therefore an example of a complex adaptive system, characterized by uncertainty and surprise [48].The adaptive cycle, which identifies phases of exploitation, conservation, release and reorganization, is a key contribution of resilience thinking, and helps to describe the ability of SESs to deal with uncertainty and surprise.Multiple adaptive cycles, which involve feedback loops and interplay between cycles across scales, are nested within a SES.The existence of such cycles has been described as panarchy [23].The adaptive cycle and panarchy can be applied to understand the resilience of SESs [21,49], in particular how SESs deal with uncertainty and surprise [48].
A key criticism of resilience thinking is that it often presents resilience as a positive attribute, yet this overlooks that a resilient system may bounce back to an undesirable state [50,51].For example, areas depleted of natural resources are often extremely resilient but may provide little in terms of income, food, or other ecosystem services.Moreover, a system with highly polluted water supplies or governed under a dictatorship may be highly resilient, but unjust, undesirable, or bear high economic and social costs [52].Defining resilience as robustness or ability to bounce back to a prior state can maintain the status quo and allow unsustainable or socially unjust practices to continue [46].In practice, system transformations may be required to overcome social and economic injustices [46].This highlights the growing normative debates that emphasize the importance understanding the resilience for whom [4] and of including transformability in our understanding of resilience.
Resilience thinking has been further criticized for its inability to adequately capture social dynamics related to power and agency [48].This is because agency, the ability of individuals to exercise a degree of choice or autonomy over their own lives, is often veiled in resilience assessments that focus on a household, community or system scale [5].For example, a household may act to strengthen their overall resilience, but at the detriment of an individual's well-being [53].Similarly, resilience assessments may take place at the system level, masking impacts at other, smaller scales.This limitation is common to all system-oriented approaches [4], where the focus is on the SES, rather than on the choices made by individuals or groups within the system [53].This suggests it is vital to highlight the importance of scale and the notion of "winners and losers" when assessing resilience [54].
In the context of climate change, actions that erode longer-term resilience may be termed "mal-adaptations" [55], defined as any response that is not sustainable or which increases vulnerability [27].In the context of smallholder farming, the sale of farm assets during drought to enable coping over the short term, for example, can undermine smallholders' abilities to re-engage in future agricultural production.Resilience assessments need to recognize that both spatial and temporal trade-offs may be encountered [38].
Resilience is also sometimes confused with development.The literature suggests that resilience can act as a barrier to development, whilst development may undermine resilience [56], through for example, the construction of houses on flood plains.Recognizing and understanding the limitations to resilience is therefore imperative.Yet, building resilience is increasingly recognized as a central development objective alongside poverty reduction and economic growth [56].
This section has demonstrated that resilience research highlights a number of considerations that should be borne in mind when using resilience thinking as a conceptual framing for climate resilience assessments.A summary of the key characteristics of resilience that emerge from our literature review is presented in Table 1.They were identified as reoccurring characteristics across multiple papers in the reviewed literature and were distilled to reflect the current state of resilience thinking.Identifying characteristics grounded in current resilience thinking provided the boundaries for this study.
The next section explores approaches to assessment, including the assessment of resilience, before focusing on the climate resilience of smallholder farming systems.Table 1.Characteristics of resilience identified from the resilience literature to reflect the current state of resilience thinking.

Key Characteristics
Indicative References 1. Holistic approaches are required to understand interactions, interconnectedness and interdependence between human and biophysical components of a single complex system [6,44] 2. Resilience is not only an antonym of vulnerability [7,25] 3. Resilience can be an intrinsic system property or process, independent of exposure to a shock or stress [33,45] 4. Resilience indicators should include social, financial, political, and institutional considerations [35,36] 5. Resilience requires flexibility, learning and transformability, where the ability of individuals and/or systems to learn from past exposure is important [31,37,48,50,57] 6.The adaptive cycle (exploitation, conservation, release and reorganization) and panarchy can be applied to understand the resilience of SES [33,49] 7. Resilience is not always a positive attribute [3,45] 8. Social dynamics and issues of power and agency should be included [3,4,48] 9. Resilience has temporal dimensions and may require trade-offs (e.g., strengthening resilience in the short term may reduce resilience in the long term) [38,55] 10.Spatial scale is important in recognizing there may be "winners and losers" and trade-offs between resilience and well-being [3,53]

Approaches to Assessing Resilience and Their Application in Farming Systems
Given the complex and dynamic nature of resilience, some researchers have suggested that it is not something that can be meaningfully quantified or measured [58].Critics consider the findings from resilience assessments to be too contextual to be useful in informing wider scale decisions.Despite this, several relevant tools aimed at assessing resilience at different scales have been developed [1] and applied across a wide range of sectors, including agriculture.The benefits of adopting a resilience framework in understanding farming and other natural resource based systems are well established in the literature [44,50,51,59] and incorporate a range of different assessment methods.This section outlines common assessment approaches.
Assessments of issues such as natural resource degradation note the importance of capturing both the social and ecological context [60] and that participatory methods can play a key role in achieving this [61].Research from around the world shows the benefits of stakeholder participation and local community engagement in generating contextually relevant information and implementing projects that meet local needs [62].Furthermore, the benefits of local knowledge and community participation in monitoring and assessing social and environmental changes, including changes in resilience, are well established in the literature [63][64][65].Indicators based on local data, developed through participatory means, provide a practical form of assessment, making it possible to identify gaps and monitor progress on the ground [66].This offers the potential for the flexible, participatory and integrated approaches required to deal with climate changes [67].Such approaches have been widely applied in the assessment of SESs [36,68].They can also foster learning which, as noted earlier, can help to build resilience [41].
Systems-approaches first emerged in the 1980s to understand and explore smallholder farming across the developing world [69,70].Assessments that employ systems-approaches recognize interconnectedness of social, economic, political and institutional processes across temporal and spatial scales.Furthermore, such approaches have also been linked to farmer participation, innovation and learning [71].Resilience thinking inspires systems-based approaches to analyzing systems with both social and ecological components, and in doing so, recognizes the importance of interconnectedness and interdependency and the context in which they are embedded [44].The benefits of bringing together farming systems approaches and resilience thinking have been established by existing studies [72,73].There is potential to build on this further to strengthen climate resilience of smallholder farming systems.
In the context of climate change and variability, climate resilience, particularly in rain-fed systems, is an important property of smallholder farming systems.Whilst general resilience may also act as a barrier to change [50,51], climate resilience may better enable smallholder farming to survive and thrive in the face of the future climate uncertainties.We define climate resilience as the resilience of a system or part of a system to climate-related shocks and stresses, i.e., the ability to survive, recover from, and even thrive in changing climatic conditions [74], and in the process, maintain essential functions, identities and structures [75].

Research Design and Methodology
This section outlines the multi-step approach we took to identifying resilience assessment tools and frameworks and explores how the key characteristics of resilience identified in our literature review have been incorporated therein (Figure 1).
Alongside the literature review of resilience research (Phase 1 in Figure 1); an international stakeholder workshop on resilience assessment tools was convened in May 2013 (Step 3 in Figure 1).International expert participants (both academics and practitioners) were asked to suggest an initial sample of relevant tools and frameworks for this research, both during the workshop and as part of an ongoing process (May-December 2013).Workshop approaches have been used effectively in previous research to bring together different knowledges, providing a valuable starting point for further work [76].
Tools that were identified by participants during and after the workshop had varying degrees of relevance for farming systems but nevertheless provided a useful insight into some of the tools, frameworks and approaches used in practical assessments.To identify further tools and frameworks, we selected search terms from the literature review (Step 4 in Figure 1) and used these to conduct a systematic internet search using a search engine (Google, November 2013).The strengths of using this kind of systematic process are outlined in the literature [77].Google was initially selected as a search engine over an academic database in order to capture those tools and frameworks that had been or are being developed and used by practitioners, not just academics.To create a manageable sample through the search process, we considered the first 20 results returned by Google that emerged during each iteration of the search terms.This yielded results from sectors as diverse as psychology and business to climate resilient cities.To narrow down the sample for further analysis and enhance its relevance to the assessment of resilience in farming systems, we next applied a set of criteria (Table 2).To be included for further analysis, tools and frameworks had to meet two or more of the criteria set out in Table 2.In total, our combined searches and the expert workshop resulted in an overall set of 25 relevant tools and frameworks.We narrowed the sample further by focusing on tools and frameworks that had supporting written documents that were available in English, resulting in 23 relevant tools and frameworks.SSA is projected to be disproportionately affected by the impacts of future climate change and variability and have a high proportion of smallholder farmers.

Evidence of use in multiple countries/used by international organizations
Provides an indication of the coverage, utility and/or acceptability of the tool.

Specific to agriculture/farm systems/climate resilience
Agriculture is an important sector, both in terms of adaptation and mitigation, and in terms of food security.Climate resilience is one way to reduce vulnerability to the uncertainties surrounding future climate change.

Evidence of peer review
Peer review implies that an attempt has been made to link theory and practice.
Using a non-academic search engine to generate results and applying a series of relevance criteria provided a sample of tools and frameworks that have been used in practice or explicitly aim to inform practical assessments.Using a systematic approach to this ensured that the breadth of tools and frameworks used in practice was captured.Not all of the tools focused specifically on resilience, but we kept them in the sample as they matched other relevance criteria.Including this wider sample provided a way to identify lessons from a wider range of relevant assessment tools.
We analyzed the content of the 23 selected tools and frameworks (Phase 3 in Figure 1), categorizing them according to their scale of analysis, their goal and to the extent to which they focused on climate, vulnerability and/or resilience.This analysis provided an overview of the characteristics of existing tools and frameworks that have been used in practice or explicitly aim to inform practical assessments.
The next step was to search for each tool/framework in Web of Science (Phase 4, in Figure 1).Evidence of academic peer review was used as an indicator of an explicit attempt to link theory and practice.By the end of this process, two frameworks remained that specifically focused on resilience.The content of these frameworks was qualitatively analyzed in relation to the key characteristics identified from the resilience literature.The selected frameworks had both been identified in the Google search or by experts, and had undergone peer review, as evident from the Web of Science search.Focusing in on these two specific frameworks which have attempted to link resilience theory and practice allowed us to highlight important recommendations for the design and implementation of more theoretically grounded assessments.If either framework did not address one of the key issues identified in the literature, we referred back to our larger sample of tools and frameworks to identify how they incorporate the issue.This process was useful in enabling comparisons across tools and frameworks.Learning from other tools and frameworks provides a means to address the limitations of current resilience assessments.

Results: Frameworks Linking Resilience Theory and Practice
The two peer-reviewed resilience frameworks that attempt to operationalize resilience thinking and that formed the basis of our analysis were those developed by Tyler and Moench (2012) [32] and Cabell and Oelesfe (2012) [20].Although one of the frameworks is not directly concerned with resilience of smallholder farming systems, which are predominantly located in rural areas, it explicitly links resilience theory and practice.Both of our case study frameworks identify and discuss elements, indicators and characteristics of resilient systems.This section presents results from the qualitative content analysis of these case study frameworks to provide insight into how resilience theory has been operationalized in practice.Comparing these with the key characteristics identified from our review of the resilience literature enables the identification of existing gaps, potential ways to strengthen links between resilience theory and practice, and, following referral back to our larger sample of tools, allows identification of recommendations for the further development of resilience frameworks and assessment tools.The section begins with a short description of each framework.
Tyler and Moench [32] developed a practical, conceptual framework for assessing urban climate resilience (Tool 11 in Appendix Table A1).The framework highlights three elements of urban resilience: systems, agents and institutions.For each of these, they propose resilience characteristics from a diverse body of literature, which they go on to describe and exemplify to develop a conceptual framework to operationalize in the context of local, urban planning.Characteristics of a resilient system are flexibility and diversity; redundancy and modularity; and safe failure.Agents in a resilient system should be: responsive and capable of reorganization, planning and responding in a timely manner; resourceful and able to mobilize assets for action; and have the capacity to learn and internalize past experiences.Institutions should: provide an inclusive structure for rights and entitlements; have transparent, representative and accountable decision-making processes; provide relevant information; and encourage the application of evidence and new knowledge.These characteristics of resilient urban systems are presented as guidelines for thinking about complex urban systems rather than technical prescriptions.The urban climate framework was found to be used by Asian Cities Climate Change Resilience Network (ACCRN), yet limited review or evaluation of the practical testing or application of this framework was found.
In the second framework, Cabell and Oelesfe (2012) compiled 13 behavior-based indicators of agro-ecosystem resilience from the literature, to include both ecological and social elements (Tool 22 in Appendix Table A1).The indicators are: socially self-organized; ecologically self-regulated; appropriately connected; high degree of functional and response diversity; optimally redundant; high degree of spatial and temporal heterogeneity; carefully exposed to disturbance; responsibly coupled with local natural capital; reflected and shared learning; globally autonomous and locally interdependent; honors legacy while investing in the future; builds human capital; and reasonably profitable.Each of the indicators is linked to phases in the adaptive cycle: growth/exploitation, conservation, release, and reorganization/renewal [23] For each of the 13 indicators, a description and a practical example is provided.
Indicators are used to determine resilience of the agro-ecosystem and can be used to monitor changes; for example an absence or disappearance of an indicator suggests vulnerabilities and movements away from resilience.The framework does not focus on assessing the resilience of the system to a specific shock or stress, and instead identifies rules of thumb to guide farmers and other stakeholders in the agricultural sector [20].It integrates indicators into the adaptive cycle, can be applied at multiple spatial scales, and is appropriate for both current and future assessments.Although the list of indicators is theoretically grounded, we found no evidence that this framework has been operationalized or tested.
The results from qualitative content analysis of the frameworks in relation to the key characteristics identified in the resilience literature are presented in Table 3. Analysis shows that both frameworks recognize the interactions and interconnectedness between the human and biophysical components of a complex system.Both frameworks conceptualize resilience as an antonym of vulnerability, and thus do not fully reflect the breadth of thinking encountered across the resilience literature.We interrogated the wider sample and found that whilst other tools that focus on either resilience or vulnerability may recognize that the relationship between resilience and vulnerability is complex, this was not captured in the indicators, elements or characteristics set out in the tools and frameworks.
Both case study frameworks partially consider financial and institutional considerations, yet political considerations are largely excluded.Whilst agency is recognized as an integral part of a system (Tyler and Moench, 2012), power relations are excluded from indicators and analysis.Tyler and Moench (2012) justify this choice on the grounds that these are not issues specific to climate resilience.We note that both frameworks provide limited space for consideration of power structures and their influence on resilience, suggesting this is a key gap that future resilience assessment tools and frameworks should address.To identify opportunities to learn from other frameworks and tools, we referred back to the larger sample and found that others explicitly incorporate considerations of power and politics, for example the Household Economy Approach (HEA) [78] and recent initiatives framed around the Resilience Assessment Workbook [22] and Climate Smart Agriculture (CSA) [79].
The HEA uses a political economy analysis to enable a deeper understanding of the causes of poverty and food insecurity [78].The tool includes questions that capture information on policies, institutions and processes and also explicitly considers political and economic interests of different actors.HEA is sensitive to power relationships and conflict in both programming interventions and when making recommendations.The Resilience Assessment Workbook also includes the identification of key formal and informal institutions, facilitates discussion about ability to influence decision-making, and the mapping of power relations and conflicts [22].The participatory planning process outlined in the CSA framework promotes change and the empowerment of local stakeholders, with a focus on the least powerful [79].Captured by multiple indicators.Feedbacks are not considered.

Resilience is not only conceptualized as an antonym of vulnerability
Resilience is conceptualized as an antonym of vulnerability.
Resilience is conceptualized as an antonym of vulnerability.

3.
Resilience can be an intrinsic system property or process, independent of exposure to a shock or stress Though climate exposure is highlighted, the conceptual framework implies that resilience can be assessed independent of exposure.
Resilience as an intrinsic system property underpins the conceptual framework and all indicators.

Resilience indicators should include social, financial, political, and institutional considerations
Political dimensions excluded.Both formal and informal institutional dimensions are considered.Financial assets included.
Political issues conceptualized as external to the system.Both formal and informal institutional dimensions are considered.Financial considerations captured in "reasonably profitable" indicator, specifically focuses on financial independence from subsidies.

5.
Resilience requires flexibility, learning and transformability, where the ability of individuals and/or systems to learn from past exposure is important.
Learning is included on an individual level not mentioned in relation to system.Flexibility is recognized as important and captured by multiple resilience characteristics.Transformability not considered.
Learning is captured on a system and individual level, by the "reflective and shared learning" indicator.Flexibility is captured by multiple indicators.Transformability forms part of the definition of resilience, but it is not explicitly captured by any of the indicators.
6.The adaptive cycle (exploitation, conservation, release and reorganization) and panarchy can be applied to understand the resilience of SES.
Panarchy is not included.The adaptive cycle is not explicitly considered, but learning and governance are recognized as important elements of enabling a system to reorganize.
Recognizes that agro-ecosystems move through four phases in adaptive cycle.Links each indicator with phase in the adaptive cycle.Issue recognized in the main text but not captured in the indicators.

Social dynamics and issues of power and agency should be included
Agency is recognized as an integral part of the system, yet power relations are explicitly excluded.
Not specified-words power and agency do not appear in paper.Actors are mentioned, but not the power relationships between them.

9.
Resilience has temporal dimensions and may require trade-offs (e.g., strengthening resilience in the short term may reduce resilience in the long term) Not included Issue recognized in the main text.It is implied that the proposed indicators overcome this.
10. Spatial scale is important in recognizing there may be "winners and losers" and trade-offs between resilience and well-being Indicators capture multi-scalar dimensions of system.Consideration of winners and losers not included.
Indicators capture multi-scalar dimensions of system.Focuses on system rather than individuals or groups of winners and losers.However, tradeoffs between indicators are noted, but it is unclear what the implications of this are for resilience.
Qualitative content analysis of our two case study frameworks shows that flexibility and learning and change are embedded in both, along with recognition that resilience can be thought of as an intrinsic system property, independent of exposure to a shock or stress.System transformability is considered by Cabell and Oelsofe [20], although they do not explicitly identify which indicators are important for adaptability or transformability.Tyler and Moench [32] do not consider transformability.The Resilience Assessment Workbook provides space to consider if transformation is needed and a way of identifying and assessing what kind of strategies may be needed to bring about transformational change [22].There is potential to learn from the processes outlined in the Resilience Assessment Workbook to enhance future climate resilience assessments.
Whilst the frameworks may provide useful ways to assess resilience and operationalize concepts such as flexibility and participation in particular systems, they do not explicitly state how they fit within a wider process of building general resilience.For example, they do not explicitly identify how different stakeholder interests and locally relevant knowledge or bottom-up approaches could be integrated.The expert-led, top-down nature of the frameworks may limit their ability to measure what is important in reality, such that contextual information is missed.Given the dynamic nature of farming systems, which Cabell and Oelsofe [20] describe as like aiming at a moving target, generating contextually relevant information to enhance our understanding of resilience is paramount if the resilience of farming systems is to be strengthened.Of the other tools, the Resilience Assessment Workbook recognizes multi-level governance and includes a process to map power relations [22], offering potential to learn from this to inform future climate resilience assessments.
Conceptually, both case study frameworks recognize the interconnectedness and interdependence of systems, and therefore take into account multi-scalar dimensions.However, limited consideration is given to how certain individuals or groups may be "winners" or "losers".Tyler and Moench [32] conclude that the central issue of "whose resilience" is not clearly articulated in the urban resilience framework.Referring back to the wider sample, the HEA focuses on social protection and identifying who is or may be marginalized, using a political economic analysis to understand underlying causes and root drivers of marginalization [78].Similarly, the Social-ecological Inventory provides a structured way to identify key stakeholders and analyze the relationships between them [80].
Neither of our case study frameworks provides a space to reflect upon the potential trade-offs, or synergies, between indicators, or their implications for resilience.Additionally, trade-offs between resilience and poverty reduction or development are not explicitly considered.Although both case study frameworks recognize that resilience is not always positive and that strengthening resilience in the short term may undermine future resilience, neither specifically addresses this when developing indicators or characteristics.Referring back to the wider sample, many of the other tools also fail to identify trade-offs.One exception is the Sustainability Assessment of Food and Agriculture systems tool, which provides a structured way to analyze trade-offs and synergies between elements of sustainability [81].This tool could usefully inform future climate resilience assessments, guiding the consideration of trade-offs and synergies in both future resilience assessments and assessments targeted at farming systems [82].
Results from the analysis of two case study frameworks therefore demonstrate that there are gaps between resilience thinking and is application through assessment frameworks and tools.Referring back to the wider sample enabled identification of potential ways to learn from other tools and frameworks in order to strengthen the climate resilience of smallholder farming systems.

Discussion: Recommendations for Assessing the Climate Resilience of Smallholder Farming System in Sub-Saharan Africa
This section draws on our findings to provide recommendations to strengthen the theoretical grounding of future resilience tools and frameworks that aim to assess the climate resilience of smallholder farming systems in SSA.The potential benefits of this have both theoretical and applied relevance.Before discussing our recommendations, it is important to recognize that other assessment tools and frameworks currently under development may have been excluded from our sample frame.We also recognize that we do not capture all of the practical lessons from relevant tools and frameworks, as our assessment was limited by the amount of data and information available online.This suggests that further lessons may be revealed should an alternative approach be taken, and this represents an important avenue for further work.
Our recommendations are as follows, in no particular order: (1) There is considerable potential to better integrate vulnerability and resilience approaches, both in theory and in practice.In the reviewed frameworks and tools, resilience is conceptualized as an antonym of vulnerability, with the links between vulnerability and resilience being overlooked or oversimplified.As such, the frameworks and tools do not capture the range of debates in the academic literature.Furthermore, the complementarities between vulnerability and resilience and the potential to bring them together are ignored in the two case study frameworks, as well as within the wider sample.Although vulnerability and resilience are rooted in different epistemological traditions in the natural and social sciences, there are overlaps in the theory, methodology, and application of the concepts [7].Adger [83] (p.269) argues that "the points of convergence are more numerous and more fundamental than the points of divergence".In practice, more consideration is needed regarding how to better identify and build on the synergies between resilience and vulnerability and how this can be integrated into tools that assess the climate resilience of smallholder farming systems in SSA.
(2) Explicit recognition is needed that resilience is not the same as development or poverty reduction.The complex relationship between resilience and poverty reduction is noted in the resilience literature [3].Strengthening specific resilience, such as climate resilience, may contribute to increasing overall or general resilience.However, in strengthening climate resilience we should not assume that poverty is reduced [53,56]; this complexity should be addressed by future tools and frameworks.This brings us back to the problem of systems bouncing back to undesirable states, where populations live in poverty, and demonstrates the need to increase capacity for transformation [31].In practice, we need to be explicit about what strengthening resilience will achieve.This requires a clear definition of what resilience is and what (and who) it includes and excludes.Further research is also required to understand the implications of resilience frameworks and tools and how labelling an individual, community or system as resilient can lead to discursive traps and be used to justify inaction [84].In light of this, we encourage further debate surrounding the relationship between resilience, development and other development goals, e.g., poverty reduction or sustainable livelihoods.
(3) Participatory, flexible and learning approaches to planning, implementation and monitoring and evaluation are important; benefits and limitations should be explicit.Resilience approaches foster learning [50], flexibility [85], participation and empowerment [57,86].Although these characteristics are considered by some of the frameworks in our wider sample, reflection on how to operationalize and measure progress is lacking.Furthermore, fostering participatory processes in practice, and benefits and limitations of participation are often neglected and should be made explicit.One possibility is to integrate processes of learning and social learning into both the development and application of resilience frameworks and tools [43].Putting this into practice may require multi-disciplinary research teams and wider stakeholder engagement [87].Smallholder farmers often have significant expertise when it comes to managing their farm system [88].There is potential to learn from farmers, and where possible, their experience and knowledge should be integrated and enhanced to strengthen the resilience of farming systems.Contextually-relevant information and locally-identified indicators provide a practical way to monitor progress and also increase the potential of generating contextually relevant solutions that can not only increase resilience but also empower farmers in the process [89].This local level understanding could contribute to Recommendation 4.

(4) Better integrate issues of power, change and transformation into tools and frameworks.
From our analysis of the two case study resilience frameworks and the wider sample, we observed that issues of power, politics and agency are underrepresented.From the wider sample, we found that issues of power and agency are explicitly considered in tools such as the HEA, which do not have such an explicit resilience focus.Learning from these tools could provide lessons on integrating such considerations into climate resilience assessments.This reduces the risk of framing climate change debates in terms of technical and apolitical solutions that ignore notions of equality, social justice and power [7,90].Pelling [46] proposes that resilience cannot be defined as buffering alone, therefore considerations of power and social justice must be considered and conceptualizations of resilience should capture the capacity for change.In order to assess this ability, certain system properties or capacities, including social, ecological and institutional components, are important and need to be incorporated into assessments.Capacities at an individual or farm level should also be considered.This links to broader questions of how to link resilience, agency and SESs [53,91].Addressing such issues could also feed into Recommendation 5.
(5) Resilience tools and frameworks targeting specific systems are needed, recognizing spatial and temporal dynamics and trade-offs.The resilience literature highlights the importance of the contextual factors that shape resilience [2].However, we identified only one peer reviewed framework specific to agro-ecosystems, and found little empirical evidence to demonstrate how relevant or useful such a framework is in practice.Whilst resilience tools and frameworks designed for other contexts, e.g., urban environments, provide some insights for linking resilience theory and practice, they have largely been tested in a limited geographical area (Asia) and may not be applicable to contextual complexities of elsewhere.We therefore highlight the need for more empirical research and testing to guide which indicators are necessary and to identify to which systems they apply.We also highlight an opportunity for future practical tools and frameworks that focus on specific systems and a space for tools and frameworks that focus on the individual, household or farm level.
Our findings further demonstrate that existing tools and frameworks fail to capture the temporal and spatial dynamics of SESs.Moreover, they do not adequately address or incorporate the spatial and temporal trade-offs that may be required in dealing with climate variability and change.Future frameworks and tools should be theoretically grounded, and also empirically tested to identify the "win wins" for resilience, i.e., the indicators or characteristics which ensure resilience across a range of spatial and temporal scales.
(6) Encourage knowledge sharing, empirical studies and critical evaluations of resilience and resilience tools and frameworks.Limited data availability on how resilience frameworks and tools are used in practice made it difficult to fully assess some aspects of existing tools and frameworks, for example their geographical coverage and application.There is scope to extend the approach used in this paper to identify further tools and frameworks, and compare the outcomes with the findings presented in this paper.For example, a search engine for academic papers such as Scopus, or different search criteria, could be used to identify additional tools and frameworks.
We found limited critical reflection or evaluation of tools and frameworks that have been used in practice.Given the complex nature of resilience, and other practical constraints such as time and funding, this is perhaps unsurprising.We nevertheless encourage practitioners and academics to carry out and publish critical evaluations of their tools and frameworks and to develop partnerships in order to facilitate learning across tools and reduce duplication of efforts.The wider resilience literature critically engages with the limits to resilience e.g., [3,4,92], yet this was missing in our sample of tools and frameworks.Such practical insights to the limits of resilience could contribute useful additional understanding to resilience thinking.
Our recommendations provide an important first-step to guide climate resilience assessments in smallholder farming systems in SSA.To build the theoretical grounding of resilience thinking, and bridge the gap between resilience thinking and assessment practice, future assessments should continue to consider the interconnections and interactions of human components and natural resources; thus building on one of the major strengths of using a resilience approach.We highlight that there is also potential to learn from other assessment tools and frameworks that address some of the current gaps in resilience assessment, for example, the way in which HEA and the Resilience Workbook incorporate issues of power, marginalization and agency.We also highlight a need to integrate the knowledge and expertise of smallholder farmers, as the major agents in smallholder farming systems.Strengthening climate resilience could be part of a wider participatory and learning process to empower smallholder farmers with the capacity for change and transformation.However, we also note that future assessments need to critically consider what resilience is, how it relates to vulnerability, development and poverty reduction, and what (and who) it includes and excludes.Incorporating such critical analysis into resilience assessments would provide scope to identify and address potential trade-offs between spatial and temporal scales; thus contributing to the advancement of resilience thinking and practice and enhanced natural resource management.

Conclusions
The range of peer-reviewed articles, frameworks and tools that mention resilience highlights its increasing popularity as an area of academic enquiry as well as goal in development practice.Due to the multifaceted nature of the concept, any attempts to assess resilience face the challenge of how to capture the dynamic and interconnected nature of the physical, social, institutional, economic, and ecological dimensions [5] of SES across spatial and temporal scales [1].In this paper we analyzed a broad sample of tools and frameworks, including resilience assessment frameworks, two of which underwent further in depth analysis.In doing this, we compared how resilience theory has been operationalized to inform the development of practical assessment tools and frameworks.Findings clearly highlight some key gaps between resilience theory and its operationalization.This demonstrates a lack of shared understanding about what resilience is, how to build it, and linked to that, how to measure or characterize it.This lack of shared understanding highlights the potential difficulties encountered when developing and implementing a climate resilience framework that simultaneously tackles complexity, context-dependency and temporal and spatial dynamics.
Through comparing resilience theory and how it has informed the development of resilience frameworks, we have proposed six recommendations for future resilience assessments.Our recommendations highlight the current real-world application of resilience thinking and represent a first step to enhance our understanding of how theory and practice can be connected.Our findings can be used inform the development of future resilience assessment tools and frameworks, and provide a potential starting point for further empirical studies of farming system resilience.For smallholder farming systems in particular, we note that specific assessment tools and frameworks are required that integrate farmers' knowledge and promote participation, flexibility and learning.This would enable assessments to better integrate contextually-relevant information and locally-held knowledge to empower smallholder farmers, which will be essential in building the resilience of smallholder farming systems.

Appendix
Table A1.Overview of existing 25 tools/frameworks identified through selection process.Those presented at the stakeholder workshop in Burkina Faso are in white boxes, those identified through the internet search in are shaded in light grey boxes and additional expert recommendations are in dark grey boxes.Tools marked with a star (*) were under development at the time of analysis and the two frameworks assessed in detail in the text are highlighted in bold.

No.
Name To enable people and companies undertaking the self-assessment to identify areas of high sustainability and areas where action is needed to improve sustainability.FAO [95] To assess land degradation at the sub-regional, regional, national and global scales.IISD [96] To systematically assess the impacts of a project on some of the local determinants of vulnerability and exposure, so that project planners and managers can design activities that foster climate adaptation (i.e., adaptation to climate variability and change).Grant [103] To provide community leaders, administrators, planners, engineers, public work directors, and/or natural resource managers with a simple and inexpensive method to review their communities potential vulnerabilities to climate trends and to begin the conversation of how and when to incorporate these trends into planning and projects within our communities.
16 ADAPT World Bank [104] A screening tool designed to bring together climate databases and expert assessment of the threats and opportunities arising from climate variability and change.17 The Resilience Tool FAO [105] To provide a framework for understanding the most effective combination of short and long term strategies for lifting families out of cycles of poverty and hunger.

18
Rapid Assessment [106] FAO [107] To assist investment project formulation practitioners in incorporating climate change considerations into agricultural investment projects and programs.

Resilience in
Social-Ecological Systems

Resilience
Alliance [22] To provide a step-by-step approach to assessing resilience of a socialecological system with the long term goal of sustainable delivery of environmental benefits linked to human well-being.To allow farmers and pastoralists to self-assess their climate resilience in order to identify areas of improvement.Results from a rapid assessment are discussed with facilitators who are provided with potential actions and guidance documents to improve resilience of farmers and pastoralists.
Includes governance, environmental, practices, social and economic questions.

Figure 1 .
Figure 1.Flow chart showing systematic approach to tool/framework identification and analysis.Processes are in white text boxes, number (n) of tools/frameworks in circles.

FAO [ 94 ]
interventions more efficient and resilient.Provide a methodological approach to analyze development measures with regard to the current and future challenges and opportunities presented by climate change.To evaluate and analyze different components of irrigation and canal systems in order to develop a modernization plan.

FAO [ 109 ]
knowledge and activities already underway in an area or sector, as well as the key actors involved with particular issues.To build the resilience of vulnerable individuals, households, communities and societies from the ground up.To develop a well-conceptualized model of resilience that can be used in both research and practice to understand and evaluate climate change and other interventions in urban settlements.To provide decision-makers with clear indications of where and how to intervene to strengthen resilience.RIMA identifies populations most in need in order to frame policy, investment and response options in terms of resilience.RIMA also enables monitoring and evaluation of the impact of interventions to achieve greater accountability towards affected populations.

Table 2 .
Selection criteria and search terms used to identify tools/frameworks.

Table 3 .
Qualitative content analysis of frameworks compared with characteristics identified from the resilience literature.Convergence between the frameworks and literature are not shaded, characteristics that are partially covered by the frameworks are shaded in light grey, and if not represented in the framework, shaded in dark grey.

Tyler and Moench [32] To build networked resilience that is capable of addressing emerging, indirect and slow-onset climate impacts and hazards.
To strengthen the long-term food security of vulnerable populations in Central America by improving the climate resilience of food systems at different spatial and temporal scales.
To provide a comprehensive understanding of personal resilience and give examples of how this could impact on users responses to demanding work situations.

Table A1 .
Cont.To assist disaster reduction efforts by the cities and local governments that has signed up to the global "Making Cities Resilient" Campaign.