Making Visible the Invisible: Why Disability-Disaggregated Data is Vital to “Leave No-One Behind”

: People with disability make up approximately 15% of the world’s population and are, therefore, a major focus of the ‘leave no-one behind’ agenda. It is well known that people with disabilities face exclusion, particularly in low-income contexts, where 80% of people with disability live. Understanding the detail and causes of exclusion is crucial to achieving inclusion, but this cannot be done without good quality, comprehensive data. Against the background of the Convention for the Rights of Persons with Disabilities in 2006, and the advent of 2015’s 2030 Agenda for Sustainable Development there has never been a better time for the drive towards equality of inclusion for people with disability. Governments have laid out targets across seventeen Sustainable Development Goals (SDGs), with explicit references to people with disability. Good quality comprehensive disability data, however, is essential to measuring progress towards these targets and goals, and ultimately their success. It is commonly assumed that there is a lack of disability data, and development actors tend to attribute lack of data as the reason for failing to proactively plan for the inclusion of people with disabilities within their programming. However, it is an incorrect assumption that there is a lack of disability data. There is now a growing amount of disability data available. Disability, however, is a notoriously complex phenomenon, with deﬁnitions of disability varying across contexts, as well as variations in methodologies that are employed to measure it. Therefore, the body of disability data that does exist is not comprehensive, is often of low quality, and is lacking in comparability. The need for comprehensive, high quality disability data is an urgent priority bringing together a number of disability actors, with a concerted response underway. We argue here that enough data does exist and can be easily disaggregated as demonstrated by Leonard Cheshire’s Disability Data Portal and other studies using the Washington Group Question Sets developed by the Washington Group on Disability Statistics. Disaggregated data can improve planning and budgeting for reasonable accommodation to realise the human rights of people with disabilities. We know from existing evidence that disability data has the potential to drive improvements, allowing the monitoring and evaluation so essential to the success of the 2030 agenda of ‘leaving no-one behind’.


Introduction
It is estimated that 1 billion people, approximately 15% of the world's population, have a disability, with 80% of this population living in developing countries [1]. Those marginalised often experience higher rates of poverty and social exclusion. Although under the Convention of The Exclusion is faced at all stages of life. Children with disabilities are excluded from school [7], for reasons such as lack of resources to ensure accessible education, and inadequate capacity to provide quality education [7]. A study initiated by the Global Partnership for Education [8] in 51 developing countries reveals that the lack of reliable data on disability is commonly mentioned as a key barrier to education for children with disabilities. In fact, across these countries there was limited information about the number of disabled children, the proportion not in school, the type of school (special schools, mainstream schools), and the facilities available, including transportation. Furthermore, public authorities in these countries acknowledged that the scarcity of data prevented the identification of barriers to education for disabled children as well as the planning and the implementation of disability-specific programs that could enhance these children's wellbeing [8].
People with disability are excluded across all arenas. They more likely to report higher rates of unmet health needs compared to non-disabled people [1]. For instance, a recent survey indicated that between 76% and 85% of people in developing countries with serious mental disorders received no treatment in the year prior to the study [9]. Disability is also associated with poor health. For example, some studies have documented higher rates of HIV among people with disabilities relative to the general population, which may reflect both disabilities associated with HIV infection and a greater vulnerability to HIV infection due to social and environmental factors (e.g., exclusion from sexual education) [10,11]. The outcome of the Disability Disaggregation Project conducted by Sightsavers in India [12] revealed that collecting data on disability promotes disability inclusive health. In the framework of this project, information about the disability status of patients were collected in health care centres using the Washington Group Short Set questions. Results show that just the process of collecting data positively influenced the attitude of health providers towards people with disabilities. In fact, the former started planning and delivering additional services to ensure the availability of wheelchairs at reception counters, as well as lifts and new ramps in facilities. [12].
Finally, even in the development sector, where the fact that people with disabilities have greater need is well acknowledged, they remain excluded from humanitarian and international aid efforts [13].
There are a range of causative factors behind the social exclusion that people with disabilities experience, in addition to the cross-cutting issues of prejudice and stigma [14]. They include lack of accessible services, such as the information that is provided in hospitals not being available in accessible forms for people who have visual impairments. Inaccessible physical environments are also an additional barrier, for example a lack of accessible public transport preventing people from being able to reach the workplace. Further compounding issues are a general lack of skills, understanding and awareness around disability. This includes professionals who directly provide services to people with disabilities such as healthcare workers and teachers [15], as well as policy-makers and people who decide the allocation of resources and design services. All of these factors are interlinked and have a complex cause and effect relationship. For example, the lower rates of employment [16] and socio-economic disadvantage that people with disabilities experience are both a cause and consequence of poverty [17] and are further compounded by widespread examples of extreme economic exploitation of people with disability [18]. Thus, there is an urgent need to redress the inequalities that people with disabilities face globally, encapsulated through the SDGs with their mantra of "leave no-one behind".
Capturing good quality (i.e., accurate, comparable) disability data is central to this aim. For example, if censuses are capturing only a small proportion of people with disabilities that actually exist in a population, disability will remain an under-prioritized and under-resourced in the eyes of programmers and planners. On the other hand, poor quality disability data may impede the evaluation of progress. For example, if a country uses a question that only identifies people with severe disabilities (e.g., "Do you have a disability?") then it can be expected that the measured gap in outcome indicators between people with and without disability will be greater than for a country that identifies a broader range of people with activity limitations, assuming that those with the most severe activity limitations tend to face more significant barriers to participation.
Data is enormously important in realising disability in the SDGs. Despite disability being a relatively new addition to international development agenda, it is a misconception that there is little data on disability. For example, a 2015 review of child disability measurement identified over 700 existing surveys or censuses from 198 countries which specifically asked questions on disability. It also found that disability data collection has actually intensified over the past few decades and that the more concerning data limitations are actually the quality and comparability of data [19]. It is, therefore, this lack of consistent and good quality disability data in accessible forms that has limited the ability to analyse issues and evaluate progress [20].
A second issue is that while data (of varying quality) is collected through a range of national censuses, surveys, studies and reviews, it is not consistently collated, disaggregated, and disseminated. Consequently, the prevalence and composition of people with disabilities in many situations is unknown.
Accordingly, there has often been a lack of knowledge about the difficulties that they face (e.g., do deaf men possess lower levels of education to deaf women). These extant data limitations reflect the low priority that disability has historically received within the global development community. Consequently, because of the dearth of good quality accessible disability data, there has been is a lack of political will to acknowledge and address the real situation of people with disabilities and the resourcing implications this brings. We recognise that the ability of states to fully act on the implications of disability disaggregated data rest not only on political will, but material factors, like budgetary constraints, which may be more difficult for low-income countries to meet. Our point is that when no such data exists within states, political will to address the inequalities facing disabled people will be restrained, compared to if states are able to obtain disability disaggregated data.
Over the past 15 years there has been a viable and growing effort at the international, national and provincial/district levels to collect data on people with disabilities [21]. As a case in point the establishment of the Washington Group on Disability Statistics (WG) by the UN Statistical Commission in 2002, an internationally coordinated effort to improve data quality and comparability of disability statistics [22,23] has ultimately raised awareness and increased the visibility of disability data on the agenda of international meetings (such as the 2017 and 2018 World Data Forum, and the 2018 Conference of States Parties). The Washington Group on Disability Statistics comprises National Statistical Office representatives from more than 120 countries and developed the Washington Group Short Set of questions. These questions are designed to identify (in a census or survey format) people with a disability. The questions ask whether people have difficulty performing basic universal activities (walking, seeing, hearing, cognition, self-care and communication) and were originally designed for use with the general population. However, the focus on functioning and the brevity of the tool mean that it can be rapidly and easily deployed in a variety of settings. The Washington Group Short Set was not designed to be used in isolation. Rather, it should be used in conjunction with other measurement tools, i.e., included within a larger survey or registration form to enable disaggregation of other measures (e.g., educational attainment, employment status) by disability status.
More broadly, disability data is increasingly being championed on the global development agenda. A notable example was at The Global Disability Summit, an inaugural event co-hosted by the UK, the International Disability Alliance, and the Government of Kenya in 2018, which brought together disability stakeholders from around the world to scale up disability inclusion efforts and provide lasting positive change for people with disabilities, in line with the 2030 Agenda. Given the gathering momentum and increase in availability of endorsed policy and measurement tools, the excuses that we cannot collect good quality data on disability or disaggregate by other indicators are no longer valid.

The Importance of How Data is Counted
While it is possible to collect good quality disability data, careful consideration is needed regarding doing this in practice. As Mont [24] highlights discussions about how to "count" disability often imply that populations can be cleanly divided into people with disabilities and people without disabilities, using a pre-specified cut-off. In fact, disability, from the perspective of functional ability, is better understood as operating on continuum [25,26]. That is, the functional ability of every person varies and whether this constitutes "disability" first depends on the context that disability is being assessed in. For example, in the domain of mobility, one person may be able to walk up a set of stairs if handrails are provided, whereas another person may not be able use any form of stairs at all. Whether these two individuals experience difficulty and, therefore, would be counted as "disabled" by the Washington Group Questions (as well as alternative methodologies) would be influenced by the environment these persons inhabit. Do they live somewhere where accessible alternatives to stairs are commonly provided (e.g., ramps)? If so, the prevalence estimates of mobility difficulties obtained through the use of disability statistics may be smaller in comparison to a context where no modifications are commonly made to stairs, even in the hypothetical scenario where the functional ability of the two populations was evenly distributed.
In light of this, we do not suggest that there is a gold standard methodology for generating an accurate estimate of disability prevalence acontextually. Nor do we intend to suggest that disability statistics are panacea for the widespread inequality and discrimination that disabled people face. Indeed, the "counting" and assessment of disability has been used to contribute to the marginalisation of disabled people in the past, for example, determining who should be subjected to forced sterilisation programmes in the early 20th century. As such, the measurement and interpretation of disability statistics should not be employed uncritically, and states and other actors must ensure to use disability data for the emancipation of disabled people, rather than their oppression.
Notwithstanding, we suggest obtaining good quality disability data regarding the SDG's can be addressed through use of the Washington Group Questions (WGQ). The questions have been tested in low-, middle-and high-income countries in every region, are internationally comparable and are easy to incorporate into existing national statistics systems [20]. These facets have been widely recognised, with over 25 Member States and UN agencies calling for the expanded use of the Short Set of questions developed by the Washington Group in 2017 [27]. However, as stated, this corpus of disability data and the learning derived from it was not readily accessible.
To understand the data quality of currently available disability data and to collate this in one place in 2018, Leonard Cheshire undertook a robust review and analyses of existing nationally representative datasets within 48 focus countries. Specifically, disability data available on indicators across five thematic areas (economic empowerment, inclusive education, stigma and discrimination and technology and innovation) were identified and analysed. As a project output, our findings were used to develop the Disability Data Portal, a website that collates this body of data in an accessible way that was launched the Global Disability Summit in July 2018 [28]. Below, we summarise the main findings of this work. Importantly, we found that that all countries had data available to enable calculation of disaggregated results for at least some of the indicators chosen for the analysis.
In considering how disability is measured to produce estimates of prevalence it is necessary to look at the methodology [29]. In this respect, the data review corroborates that there is no consistency amongst the data sets and found that countries tend to fall into five categories of questioning. The first group poses questions such as "Do you have a disability?" which lead to underreporting because the term disability is usually associated with shame or reflects the fact that people tend to assume disability is a severe condition, so more moderate types of disability are missed. Additionally, older people may under-report as they perceive their functional limitations as an inevitable consequence of ageing as opposed to having a disability [30,31].
The second group were datasets that use medical diagnoses as a definition of disability, as opposed to functional limitations. This approach is again likely to lead to under-reporting as not all disabilities can be defined by medical impairment, and those with less education or access to diagnostic services may not be aware of their condition [32]. Furthermore, data could be inaccurate as people with the same diagnosis may have different levels of functioning or severity of limitations [33][34][35], which will not be picked up by measurement of diagnosis alone.
The third group of countries ask about functional limitations (i.e., relating to individuals who have difficulty 'doing' activities). This is preferred to asking for a medical diagnosis because it addresses what a person can or cannot do, as opposed to the reason why they cannot do it. It also avoids the requirement for a healthcare worker to provide a medical diagnosis.
The fourth group use the Washington Group Questions (WGQ) but do not strictly follow the Washington Group guidelines when administering the tool. Examples include using non-neutral language and explicitly referring to disability within questioning which is likely to generate inaccurate answers. Another example of breaching the guidelines has been to change the questions from a scaled response to a binary 'yes/no' answer. This renders the questions insensitive to the dynamic, complex process of disability, a phenomenon that is essential to capture in order to gain truly reflective data. The fifth and final category describes countries that used the Washington Group questions correctly.
When measuring prevalence a comparison of the different data sources revealed that those countries that used the WGQ as their measurement tool, generally record higher rates of disability [36]. The lower rates were recorded from countries that used questions that referred directly to disability and asked for a medical diagnosis. For example, in Zambia the measured prevalence of disability rose from 2.7% to 17.8% once the question "do you have a disability?" was replaced by the Washington Group Short Set [37]. If surveys are capturing a smaller proportion of people than that which actually exists, significant numbers of people with disabilities will remain invisible, impacting on the programming and planning needed to maximise the well-being of this population.

What Disaggregating Data Tells Us
The estimation of prevalence rates and disaggregation of indicators by disability are important for identifying areas of exclusion and subsequently supporting efforts to monitor and evaluate progress towards eliminating barriers, including those highlighted in the SDGs. This is because a more accurate understanding of the reality of marginalised populations will allow for better planning and budgeting for reasonable accommodation. If people with disabilities remain invisible in data, they remain unaccounted for. For example, the Disability Data Portal reveals that data on disability is more widely available in countries such as Cambodia, Rwanda, Timor-Leste and Uganda compared to in Myanmar or Pakistan. Moreover, it shows that disaggregating indicators is possible, and has been implemented across certain countries. In Uganda, for example, data was available to disaggregate 14 of the 16 indicators investigated.
Data on girls and women with disabilities is essential to understand the double discrimination and intersectional inequalities they experience. Women are at an increased risk of becoming disabled because of ongoing gender inequalities due to inequities in economic status and access to healthcare; in addition women live longer on average than men which may put them at higher risk of age-related functional limitations [38]. Vulnerability to intimate partner violence also increases the risk of becoming disabled [39] as can early pregnancy and poor reproductive health. It is estimated that more than 30 women every minute are seriously injured or disabled during labour [40]. Moreover, evidence shows that women with disabilities face unique, individualised barriers compared to both men with disabilities and non-disabled people. For example, Smelzer [41] documents that while disabled women face the same uncertainties around pregnancy as all women, they are also subject to unique harrowing experiences, such as receiving unsolicited advice for termination from healthcare professionals. Lastly, women and girls with disabilities are vulnerable to forms of abuse, such as forced marriage [42] and sexual abuse [43,44], and people with disabilities are generally from sexual education, which can obstruct the recognition of abuse as well as the seeking of support (e.g., reporting abuse to the authorities). Understanding how issues like abuse affect people with disabilities, and particular categories of disabled people like women, or adolescent girls, is key to designing effective prevention strategies to engender autonomy and reduce risk of harm. All of the above are important considerations to take into account when examining the reasons for inequalities and what specific interventions are required to overcome the exclusions at the intersection of gender and disability. The Disability Data Portal provides insight into these links, for example by showing that in general, the percentage of women with disabilities is slightly higher than that of men with disabilities. This indicates that policy and programming should consider the specific risks women and girls face, and the targeted interventions they need for care and support.
Inclusion in education is an important and vital first step to inclusion in general [45]. Children with disabilities who have had limited access to education often face many challenges accessing work due to missing out key educational milestones [46]. A significant number of children with disabilities are not enrolled in primary education due to stigma or lack of appropriate adaptations. This is supported by findings from the Disability Data Portal where in general results showed that people with disabilities perform less well in educational attainments than people without disabilities. In relation, the exclusion from employment that people with disabilities experience may lead to lower incomes and further exclusion from mainstream society due to an increased risk of poverty. Moreover, those with care responsibilities for family members with disabilities may forgo work opportunities, and this care work is disproportionately done by women as women carry out the majority of care responsibilities in many contexts [47,48]. In general, findings from the Disability Data Portal confirm higher rates of unemployment for people with disability in the majority of countries analysed, and further disaggregation shows that people with disabilities are more likely to be living in poverty than those without disabilities. Disability data is also useful to show that disability is not the only contributing factor to inequality. In some countries, such as Burkina Faso, the poverty gap between people with and without disabilities is relatively small, pointing to wider development issues other than disability alone. Gender is also another intersectional factor at play, with a gender-based comparison showing that in some countries studied the average employment gap between adult men with and without disabilities is twice the gap between adult women with and without disabilities. Taken all together, this suggests that disaggregating data can help us understand the reasons for inequalities which will support effective programme design; for example, disability inclusive employment will be difficult to achieve without improving education opportunities for people with disabilities.
Stigma and discrimination touch on every aspect of people with disabilities' lives. Although it is challenging to directly measure this by data disaggregation, some examples of measurable discrimination show that people with disabilities are found to be on lower pay-scales for performing the same work as individuals without disabilities [49]. This discriminatory gap upholds existing inequalities and prevents progress...
The exclusion of people with disabilities from digital infrastructure and accessible ICT may lead to people with disabilities being denied access to basic information and services. For example, a study found that technology had the potential to improve the quality of people with disabilities' daily functioning, and their capacity for independent living such as increasing employment skills. However, it also found that educational systems were failing to cater for the needs of learners with disabilities, including in the use of ICT [50]. This failure to harness the transformational potential of ICT continues to marginalise people with disabilities. In this respect, the Disability Data Portal reveals information on technology that is integral to the effective design of programmes that involve technology. For example, it shows how a greater percentage of people with disabilities own mobile phones compared to those that can access the Internet, with Internet access for people with disabilities generally low, but even lower amongst women compared to men with disabilities. This suggests that designing programmes directed at women with disabilities, that rely on internet access would be relatively ineffective.
Several reasons could explain the fact that people with disabilities are less likely to access internet services than mobile phones. Firstly, on average the financial cost associated with Internet usage is higher than that of mobile phone ownership. Secondly, Internet usage generally requires basic reading and writing skills while even those who are illiterate can easily use mobile phones. In reference to these points, people with disabilities globally are poorer and less likely to have functional literacy skills compared to people without disabilities.
All of these examples illustrate the multiple and intersecting forms of exclusion and discrimination that people with disabilities experience. The interlinked relationship between disability and poverty has been well recognized [51] and the interlinkages between education, poverty and other drivers of development can be seen as major factors contributing to bridging the access gaps between people with and without disabilities. Disaggregating data can help us understand this complex picture more clearly and enable smarter and more responsive planning and budgeting to support progress towards the SDGs by helping people with disabilities to live independently, free from violence and with dignity.
Our Data Portal currently disaggregates data for each indicator by disability status and then aggregates these data with gender, where possible. That is, through this process, we are able to capture, for example, the percentage of disabled women who are employed in Colombia, relative to the employment rate of both disabled men and non-disabled women and men. At present, the Data Portal highlights many data gaps that require addressing and it is envisaged that more data will be added over time to better highlight intersectional inequalities (e.g., ethnicity). As such, the Data Portal is a starting point and a call for action regarding the collation of disability data.

Ways Forward to Inclusivity
Reviewing our experiences in assembling this large dataset for the Data Portal, we have also come to an additional set of conclusions regarding disability data and evidence. A growing body of data and evidence already exist to illustrate a picture of the exclusion and marginalisation facing people with disabilities. However, to be able to monitor progress in overcoming these challenges along with the necessary interventions and protections the global community requires data that is up-to-date, accessible, comparable and disaggregated to highlight trends and differences. Additionally, it is paramount for Disabled Peoples Organisations (DPOs)-organisations run by and for people with disabilities-as representative bodies of the disability community, to advocate for greater availability of data to ensure deeper understanding around the barriers stopping people with disabilities from accessing basic services and enjoying equal rights. By improving and harmonising data collection methods, such as using the Washington Group questions, comparisons can be made that can facilitate the sharing of lessons learnt and the monitoring progress. The Washington Group questions can be easily and cost-effectively integrated within existing national data collection systems. Practical issues around ensuring surveys are available in easy read versions for people with intellectual disabilities or in accessible forms, such as in Braille, will ensure a higher response rate from people with disabilities as will working with DPOs to reach more people with disabilities. Additionally, training enumerators and accurate translation of the questions have a positive impact on the robustness and comparability of data between surveys. This includes building the capacity of national statistics office to be disability inclusive, such as through including DPOs. DPOs should be included and engaged in all SDG-related work at national and sub-regional levels from data-related work to programme design and to support this budget must be allocated to support DPO participation.

Conclusions
Evidence-based data on people with disabilities at the national and global levels is vital in identifying the challenges that people with disabilities face. It can support policy-makers to address gaps and amend existing policies and regulations to ensure disability inclusion and mainstreaming. The Disability Data Portal shows that existing mainstream data and indicators can be used to disaggregate data by disability.
Disability data supports policymakers to address gaps and amend existing policies and regulations to ensure disability inclusion and mainstreaming. The Disability Data Portal shows that it is no longer the case that policy-makers can attribute lack of data as a reason for preventing policy change towards meaningful disability inclusion.
Furthermore, beyond disaggregation the data process should be inclusive and transparent. To achieve this, data collection tools need to be adapted to the implementation of the SDGs.
Recent developments in technology, such as using mobile technology to collect data can create an opportunity to enhance statistics for accountability and participatory monitoring. By adopting the framing associated with the SDGs, opportunities will be created for civil society, including DPOs to engage with governments on the allocation of resources, on facilitating the implementation of the SDGs and on monitoring activities related to this process. Civil society also has the important role of directly monitoring the local implementation of the SDGs and highlighting the situation of the poorest and most marginalised.
Finally, the undeniable message of the corpus of disability data is that people with disabilities are being left behind across key indicators in the majority of countries [33]. People with disabilities are central to reaching the commitment of the global community to 'leave no-one behind', alongside policy-makers, and civil society organisations, and together they must ensure that disability inclusion is at the forefront of policy and programme implementation with disaggregation of data by disability as an essential tool to monitor and evaluate change.