1.1. Incentives and the SDG Monitoring Framework for Sanitation
The 2030 Agenda for Sustainable Development, which sets out 17 Sustainable Development Goals (SDGs) and 169 targets, was adopted by the UN Member States in 2015. Target 6.2 of the SDGs calls for universal access to sanitation by 2030, with the associated indicator 6.2.1 being the “proportion of the population using safely managed sanitation services” [1
]. The WHO/UNICEF Joint Monitoring Programme (JMP) is responsible for monitoring global progress towards this target, using a sanitation service ladder to benchmark household services as either ‘safely managed’, ‘basic’, ‘limited’, ‘unimproved’, or ‘open defecation’ [1
The explicit intention of setting global goals is to shape the development agenda, by defining priorities and creating incentives for action. At a national level, research has shown that the right incentives are imperative to drive progress towards universal sanitation services [2
]. Specifically, ‘values-based incentives’ (for instance, around modernity and cultural heritage) and ‘instrumental incentives’ (such as career progression and political return) are found to influence performance at various layers of governance and service delivery [3
]. The SDG targets and indicators can influence both these forms of incentive. First, they shape global norms and define what should be considered a ‘modern’ or ‘desirable’ sanitation service. Second, they create a framework of indicators, against which the work of individuals and institutions will be judged. However, as well as creating positive incentives, the potential also exists to trigger perverse incentives. The impact of the SDGs as a tool drive sanitation progress will largely be determined by the extent to which positive incentives are maximized and perverse incentives are limited.
To understand the ways in which the SDG targets and indicators may shape the incentives and priorities of national governments and international donors, it is useful to consider experiences from the Millennium Development Goals (MDGs). A survey of government officials and civil society organizations from across 126 countries found that the MDGs were “moderately influential” in setting national developmental priorities, with the greatest influence being in Sub-Saharan Africa [4
]. The main motivations for national governments to engage with the goals appear to have been increased global visibility and influence, and increased allocations of overseas development assistance (ODA) [5
]. Whilst the overall impact of the MDGs on development partner priorities was more mixed, there is evidence that the global targets influenced the sectoral allocation of development spending in the early years of the MDGs; MDG-linked investments increased by 76 per cent between 2000 and 2005, compared to a 46 per cent increase for non-MDG investments [4
]. While it could be argued this increase represents a correlation rather than a causality, evidence from within individual sectors reinforces the view that development spending was increasingly highly concentrated in areas that were specified in the MDG targets. For instance, the volume of ODA for primary education more than quadrupled between 2000 and 2008, whilst the volume of ODA for secondary education barely changed over the same period [6
The political process of setting global goals and targets can therefore be seen to influence development priorities by shaping both values-based and instrumental incentives. However, instrumental incentives are also shaped by the technical process of defining and measuring specific indicators. In view of this, global indicators have been framed as a “technology of governance” [7
]. According to this understanding, indicators can shape incentives in two ways. First, and most straightforwardly, indicators define the technical performance standards against which progress can be monitored. Those who are ‘monitored’ are then incentivised to improve their performance as measured by the indicator, rather than through a broader conceptualisation of the relevant goal or target. Second, indicators have a “knowledge effect”, whereby they can come to define the concepts that they were originally intended to reflect [7
]. In doing so, global indicators can lead to the establishment of “policy paradigms” in which certain policy options are elevated, and others diminished, as a result of how the indicator has been defined [9
]. In this way, the SDGs risk the development of perverse incentives, as the process of translating complex realities into globally comparable, quantifiable indicators can obscure contextual differences or local peculiarities [8
The ambition inherent in the core indicator for SDG 6.2 creates positive incentives, but also potentially perverse incentives. The inclusion of the safe management of excreta in the indicator puts a greater focus on public health, and will incentivise vital investments in faecal sludge management and wastewater treatment. However, having a single ‘top of the ladder’ indicator could create perverse incentives by obscuring the progress made at lower rungs of the ladder. Understanding how these perverse incentives could be avoided requires an appreciation of how global accountability is conceptualised and operationalised within the SDG framework.
1.2. Global Accountability within the SDG Framework
The targets and indicators of global goals only have an impact on national priorities and incentives if there is some form of accountability for their achievement. The text of the Agenda 2030 declaration states that the SDGs will promote accountability to citizens via a “robust, voluntary, effective, participatory, transparent, and integrated follow-up and review framework” [10
]. The global architecture for follow-up and review is centred on an annual high-level political forum (HLPF), which is tasked with assessing progress, achievements, and challenges, and ensuring that the agenda remains relevant and ambitious. The HLPF itself is designed to be the culmination of a network of global follow-up and review processes, which include thematic reviews of progress—these are linked to the annual theme of the HLPF and they build on the work of the Economic and Social Commission—and the Voluntary National Reviews by member states [11
]. However, whilst institutional frameworks exist to track and assess progress at the global level, the conceptualisation of ‘global accountability’ within the SDG architecture remains rather amorphous.
The difficulty of defining global accountability is by no means unique to the SDGs, posing challenges in numerous areas of international governance [12
]. Accountability at the national level is defined in terms of answerability, enforceability, and responsibility [13
]. However, these dimensions are mostly absent at the global level, and none are fully operational within the global SDG framework. Answerability and responsibility are undermined to some extent by the ‘aspirational’ nature of global targets, and also by the framing of the agenda itself. The Agenda 2030 declaration holds national governments accountable for national targets—which only need to be ‘guided’ by the global ambition—with global accountability explicitly subordinate to national processes [10
]. It has been argued by Engebretsen et al. [14
] that because of the extremely broad framing of Agenda 2030, responsibility becomes both all-encompassing and non-existent—the SDGs are “everyone’s business but no-one’s major responsibility”. These ambiguities weaken responsibility and answerability in the global accountability processes. Furthermore, enforceability is completely absent at the global level, with the follow-up and review processes designed to be voluntary in nature [11
Because of these challenges, global accountability within the SDG framework has been based largely on the principles of ‘mutual accountability’ [15
]. As articulated in the Paris Declaration on Aid Effectiveness, and later in the Busan Global Partnership for Effective Development, mutual accountability refers to a set of voluntary commitments that rely on trust and partnership to drive progress around shared agendas, rather than on sanctions for non-compliance [16
]. Within a mutual accountability framework, global mechanisms seek to harness the “power of reputation” by identifying countries or cities that can demonstrate significant progress toward meeting individual targets, and facilitating the sharing of these success stories to inspire action by others [12
]. Furthermore, accountability mechanisms should provide high quality analysis to help decision makers to better understand the possible pathways to success.
Within the follow-up and review process, there should therefore be space for in-depth qualitative assessments of progress to facilitate peer learning and course correction, as envisaged by the Voluntary National Reviews (VNRs). Analysis of the reviews submitted to date has largely argued that whilst this process has demonstrated governments’ political commitment to the SDGs, it has not yet succeeded in providing a platform for peer learning and course correction. The lack of standardisation in content has made it difficult to record and compare the progress across countries, or to identify trends and lessons of success or failure [18
]. Whilst there is clearly scope to improve the VNRs, a detailed discussion of these processes is beyond the scope of this study. Rather, this paper analyses the quantitative indicators that are used to monitor SDG 6.2 at a global level, and discusses how they can be tailored to better meet the needs of mutual accountability. It is important that these indicators are sensitive to the ambiguities of accountability that are inherent in the SDG agenda itself, and that they are able to provide the “light political tracking” needed to maintain political attention, demonstrate progress over time, and create incentives for all countries to engage, regardless of their circumstances or current levels of sanitation coverage [14
1.3. The Opportunities and Risks of the Global Monitoring Framework for SDG 6.2
The current sanitation indicators support the objectives of mutual accountability, to an extent. The strength of the JMP service ladder is its comparability across countries and its relevance for different levels of sanitation development, reflecting the gradual and often stepwise nature of progress. The ambition of the target reflects the normative criteria of the human right to sanitation, and it has stimulated discourse and action around the management of faecal waste beyond initial containment. Furthermore, the exact framing of target 6.2, which specifies the elimination of open defecation, helps to maintain a focus on both ends of the service ladder. In these ways, the global monitoring framework contributes towards the development of positive incentives for actors in the sanitation sector.
However, there is also a risk that governments and development partners may be faced with perverse incentives to concentrate efforts solely on safely managed services, to demonstrate their contribution to the global target. This creates a danger that those who already have access to basic sanitation services will be prioritised over those currently at the lower rungs of the ladder, for whom reaching safely managed services is more difficult and expensive. For instance, there are already accounts of reduced appetite for promoting and investing in shared sanitation—classified as a limited service—which is widely recognised as an important service option in certain contexts, such as dense informal settlements [21
]. These perverse incentives stem from a ‘knowledge effect’ created by the revisions to the JMP’s service ladder following the adoption of the SDG targets. As the custodial agency of SDG 6.2, it was the JMP’s responsibility to define what was meant by the term “safely managed service”. The original proposition of the JMP’s ‘Sanitation Task Team’ was to include a benchmark for ‘basic sanitation’ that would include households using facilities shared by no more than five families and by no more than thirty people. However, this recommendation was rejected by the JMP because the household surveys upon which they base global estimates do not often contain data on the number of households sharing a facility. Furthermore, they found it difficult to define an adequate proxy indicator, as the evidence on the relationship between the number of households sharing facilities and their ‘safety’ was judged to be inadequate. Consequently, the JMP excluded shared facilities from the definition of ‘basic’ and ‘safely managed services’, and created the new category of ‘limited service’ [21
]. While this decision can be seen as justified from the perspective of global monitoring, it is important to note that it has produced a policy paradigm in which shared sanitation is seen as less desirable in every context. Yet this paradigm stems from what was technically feasible from a monitoring perspective, not necessarily what was appropriate from a policy perspective. In many ways, this reinforces the criticisms made of the JMP’s ‘technology-based’ monitoring under the MDGs, and the argument that monitoring should instead focus on the intended ‘functions’ of a sanitation service [22
]. With regard to shared sanitation, this is problematic, as there are numerous examples of contexts in which household sanitation is not feasible, and—short of rehousing—high quality shared facilities represent the best option to improve service levels [21
]. Owing to this policy paradigm, the perverse incentives associated with the global indicator for SDG 6.2 are therefore most likely to penalise underserved and vulnerable populations, especially those living in dense informal settlements in the poorest countries.
A second risk of the global monitoring framework relates to accountability. This stems from the difficulty of comparing rates of progress across countries and over time in a way that accounts for changes on each rung of the ladder. As countries have domesticated the sanitation targets, many have included targets for both safely managed and basic services. National governments are primarily accountable for these national targets, and national accountability mechanisms are therefore central to their achievement. These national processes should be supported and reinforced by efforts at the global level. With regard to sanitation, the first major accountability challenge is currently the lack of effective mechanisms at the national level, as demonstrated in two recent global reviews of accountability in the sector [25
]. However, issues also exist within global mechanisms. As the headline global indicator only tracks safely managed services, it can only paint a partial picture of accountability where the national targets include specific objectives for lower levels of service. While the full JMP service ladder provides a more nuanced understanding than the headline indicator, it can be challenging to compare rates of progress, across countries and over time, in a way that accounts for changes at each rung of the ladder. The service ladder implies a form of ordinal utility, whereby each successive rung of the ladder is preferable to the previous one, but no judgement is made about how preferable. As such, it is difficult to assess progress over time in a consistent and comparable manner. For instance, which has made the greater progress: a country with a five percent increase in safely managed services, or a country with a 10 percent increase in basic services? These types of comparisons between countries and over time are central to operationalising the concept of mutual accountability that is outlined above. For global accountability mechanisms to reflect the progress made at a national level and to drive improvements in the sector, it is important that they use data that can facilitate holistic comparisons, capturing progress at each rung of the ladder. This is especially important in order to showcase examples where progress at lower rungs of the ladder has been accelerated in order to reach the poorest first.
1.4. Reframing Progress through a ‘Total Service Gap’
The two main risks posed by the global monitoring framework both therefore relate to the relative desirability of different service levels. Mechanisms of mutual accountability are undermined by the difficultly of comparing progress over time in a way that consistently accounts for changes made at different rungs of the ladder. Perverse incentives are created by a paradigm that diminishes the value of shared sanitation as a policy option. We argue these two issues could be partially addressed by reframing how ‘progress’ is defined and communicated.
This paper proposes an approach to combining the data on the SDG service ladder into one composite measure, in order to calculate a ‘total service gap’. Rather than assessing progress by the percentage of the population using any individual level of service, the data contained within the service ladder is combined into one single metric to demonstrate how far away a country is from universal safely managed services. A service gap of 0% would signify universal coverage of safely managed services, whereas a service gap of 100% would signify universal open defecation. As a country’s sanitation coverage increases, the service gap will reduce, according to relative progress on each rung of the service ladder.
Other options for strengthening monitoring of SDG6.2 have been proposed in the literature, such as those that measure sanitation from a ‘multidimensional’ perspective [27
] or from a service-oriented perspective [29
]. Both these alternatives produce a revised ladder, with additional dimensions and parameters to be measured. In doing so, they are able to capture more elements of the normative criteria of the human right to sanitation and have scope to be of value if introduced at a national level. Specifically, the multidimensional approach to measuring sanitation poverty, proposed by Gené-Garriga and Pérez-Foguet [28
], identifies associations between different ‘sanitation deprivations’, which have the potential to be of value to national and sub-national decision makers, and to improve the targeting of sanitation policies. In this respect, if adopted at a national level, these alternatives would help to prevent the emergence of the perverse incentives associated with the more simplistic indicators that are used to monitor SDG 6.2. However, the amount of additional data that are required by these approaches means that they would have limited utility for mutual accountability or cross-country comparison at the global level in the short- to medium-term, as these data are not commonly collected across countries. The total service gap, on the other hand, takes the strengthening of mutual accountability as a foundational objective. As such, a key consideration is that it can be calculated immediately, using existing global datasets. In this way, its utility for national decision-making may be lessened in comparison to alternative proposals, but it is better able to address the risks identified within the global SDG agenda.
Assessing progress through a total service gap has the potential to address the challenges that are identified within the global monitoring and accountability framework. The progress of countries could be compared across time while accounting for changes at each rung of the service ladder in a more consistent manner, feeding into stronger mutual accountability mechanisms. A composite measure of progress would limit perverse incentives by helping to create a broader policy paradigm that values lower levels of progress, such as shared sanitation. By reframing progress in terms of the ‘distance to universal services’, the total service gap provides a useful communications device to help maintain political attention on sanitation in a crowded global accountability space. Importantly, this can also help to draw attention to various forms of inequality, the elimination of which is a key tenant of the SDGs. The total service gap aims to add value to existing metrics, avoiding the creation of an additional measure of ‘coverage’ that may confuse the existing discourse around levels of ‘safely managed’, or ‘at least basic’ services.
Despite the potential benefits of such a measure, the methodological challenge of setting appropriate weights for each service level could undermine its effectiveness or impact. Ideally, the weights would be set using evidence about the relative benefits of each service level. However, there is currently insufficient evidence in the literature to be able to do this objectively. Previous studies have assessed the impact of drinking water and sanitation on diarrhoeal disease [30
], reviewed the relative health outcomes of shared and household sanitation [31
], and compared the direct health gains from household sanitation, with the external benefit of neighbourhood sanitation [33
]. However, no existing study provides the quantitative evidence that would be required to set weights for each service level on the basis of the relative health benefits. Furthermore, setting weights based on health impacts alone would be reductionist in itself, as it omits the important non-health benefits of sanitation [34
]. As such, in calculating a total service gap, the weights afforded to each service level must be set subjectively. Under the framing of a total service gap, setting the weights for open defecation and safely managed services is non-contentious. Open defecation represents a complete absence of sanitation service, and safely managed services are the international gold standard; the service level weights can therefore be set at 1 and 0, respectively. The most straight-forward way to set the weights for the remaining three service levels would be to assume a uniform increase in benefits as one moves up each rung of the service ladder: a weight of 0.75 for ‘unimproved’, 0.5 for ‘limited’ and 0.25 for ‘basic’. The strengths of using uniform weights are conceptual simplicity and ease of communication, which would be beneficial, given the political issues that the total service gap is intended to address. However, this model could be criticised for not accurately representing the relative benefits of each level of service, and therefore painting a misleading picture of progress. Weights could instead be set by expert opinion to better reflect the relative benefits of each level of service, or through the use of multivariate techniques to construct a more empirically grounded model. However, the increasing complexity that is required by each of these options could make the communication of the total service gap more challenging, and ultimately reduce its ability to address the risks outlined above.
This paper examines whether using uniform service level weights is an appropriate method for constructing a ‘total service gap’. It does so through a sensitivity analysis, comparing this approach to three ideal-type models to assess the extent to which altering the weights impacts the total service gap, and therefore our understanding of countries’ relative performance. It goes on to examine the extent to which assessing sanitation progress through the lens of a total service gap can help to address the risks associated with the global monitoring framework for SDG 6.2, with specific reference to strengthening mutual accountability and tackling perverse incentives.