Article 31 of the United Nations Convention on the Rights of Persons with Disabilities (CRPD) mandates that ratifying states “collect appropriate information, including statistical and research data, to enable them to formulate and implement policies to give effect to the present Convention” [1
]. Currently, however, there is a lack of consensus on how to measure disability, and disability prevalence estimates are still strongly influenced by each country’s conceptual or legal definitions of disability and by the questions used to operationalize these definitions in disability, health and social surveys and censuses [2
]. Such conceptual and definitional variability has not only an immediate impact on disability estimates, but may contribute in the middle and long run to inconsistent or insufficient policy solutions and, ultimately, negatively impact the lives of those experiencing disability. A recent review of disability surveys carried out in all world regions shows that the definition proposed in the WHO International Classification of Functioning, Disability and Health (ICF), as the outcome of “the interaction between an individual (with a health condition) and that individual’s contextual factors (personal and environmental factors)” [4
], has been widely adopted in disability surveys over the past few years [5
]. However, countries also tend to tailor this broad definition to their needs and to the goals of specific data collection efforts, such as surveys targeting eligibility for social programs. This might be done, for example, by introducing additional criteria specifying that a person will be considered disabled only if he or she experiences limitations that have lasted longer than one year. These country-specific additional criteria are usually operationalized in filter questions used to select participants for the survey and have an important impact on disability estimates and the comparability of these estimates across countries.
The conventional approach to collecting information on disability is to screen the population at the outset to identify ‘people with disabilities’ and then to ask this sub-population follow-up questions about everyday problems that they face. Screeners may either be impairment or functioning questions [2
]. Impairment screeners are used to select a population of disabled individuals either by asking about the presence of an impairment or health condition, while functioning screeners target limitations in selected functioning domains because of a health condition, such as problems performing activities of daily living. The short set of questions developed by the Washington City Group (WG-6) as a general disability indicator for censuses and national surveys was not originally recommended to be used as a disability screener (WCG-Recommendations for the 2010 Round of Censuses). This set has, however, been frequently so used, as it is contains six standard basic functioning domains: seeing, hearing, walking or climbing steps, remembering or concentrating, washing all over or dressing and communicating [6
]. The WG-6 was used to select respondents with disabilities in a series of surveys in several African countries about the living conditions of people with disabilities. Respondents answering at least two questions with “some difficulty” were included in the sample of disabled persons of the survey (http://www.sintef.no/home/projects/sintef-technology-and-society/2006/studies-on-living-conditions/
One of the most troublesome issues with the standard approach of using screeners is that the suitability of resulting surveys to monitor the implementation of the CRPD is questionable. Article 1 states that “Persons with disabilities include those who have long-term physical, mental, intellectual or sensory impairments which in interaction with various barriers may hinder their full and effective participation in society on an equal basis with others” [1
]. The CRPD definition stresses the rights of persons experiencing disabilities to participate in society on an equal basis with others, thus requiring an unbiased and direct comparison of the participation of persons with disabilities with the participation opportunities of the general population. This is not possible if only persons with disabilities are included in a disability survey.
In response to calls for improved disability data collection, the Model Disability Survey (MDS) project (http://www.who.int/disabilities/data/mds.pdf
) was initiated by the WHO and the World Bank (WB) in 2011. The MDS is based on ICF and represents a revolution in the concept of disability measurement. The MDS conceptualizes disability as an outcome of interactions between a person with a health condition and environmental and personal factors, rather than just focusing on a person’s health state, impairments or functional limitations. As in the ICF, the MDS defines disability as a universal phenomenon experienced on a continuum from low to high levels of severity. The MDS utilizes a general population sample without screeners or filters. The aims of the MDS are to (1) achieve comparable and standardized prevalence estimates across countries, (2) provide the data needed to design appropriate interventions, programs and policies for persons with mild, moderate and severe levels of disability and (3) provide data needed to monitor the implementation of the CRPD by allowing for a direct comparison among people with disabilities of any level of severity and those without.
Although the shortcomings of screeners have been repeatedly reported, giving them up may be challenging for countries with a long tradition of using them. Analytically demonstrating the impact of using screeners on disability rates can support countries in their decision to move on to more inclusive strategies. Providing a direct comparison between people identified as disabled and those left out after applying a screener can call attention to the risk of excluding people in need from targeted interventions, programs and policies that could improve their life situations. Furthermore, attention has to be paid to those with mild and moderate levels of disability. Policy makers need to be made aware that that these persons are at risk of developing higher levels of disability and that their needs are as significant as those of the more easily-identifiable persons with severe disabilities. Interventions targeting this group with mild and moderate levels of disability would have, overall, a higher population impact than interventions for the severe only, measured in DALYs improvement across the population.
In the present work, we used the first pilot study of the WHO Model Disability Survey that was carried out in Cambodia to address three objectives:
to analytically demonstrate the impact of disability screeners on disability rates,
to challenge the usual a priori exclusion of persons experiencing mild and moderate levels of disability from disability surveys and
to analytically demonstrate the advantage of using an a posteriori cut-off in a general population sample to identify persons experiencing disabilities.
In an effort to support the use of disability data collection strategies suitable for monitoring the CRPD, this study aimed to analytically demonstrate both the impact of disability screeners on disability rates and the advantage of using an a posteriori cut-off in a general population sample to identify persons experiencing disabilities. It is important to stress, however, that we used data from a pilot study of the Model Disability Survey in Cambodia, which included a non-representative, convenience sample of 500 persons. Consequently, reported disability rates are exemplary estimates for this convenience sample and must not be taken as official disability rates for the country. Our results show that disability screeners, i.e., screeners applied to a population at the outset to identify ‘people with disabilities’ for a survey, lead to imprecise disability rates and failed to correctly identify more than 20% of persons with severe disabilities. Importantly, this procedure leaves about 70% of persons already experiencing moderate levels of disability and nearly all persons reporting mild disability levels out of disability surveys. The use of an a posteriori cut-off and a general population sample leads to a more precise estimation of the prevailing rate of severe disability. This approach also discloses the magnitude of performance problems experienced by persons with mild and moderate disabilities and, therefore, challenges the usual a priori exclusion of these persons from disability surveys used to monitor the implementation of the CRPD.
A fundamental difference between the two approaches is their underlying assumption on how difficulties in functioning domains should translate into disability rates. The screener approach assumes that difficulties in different functioning domains are directly comparable, i.e., that persons stating they have, for instance, important difficulties in communicating and hearing or important difficulties in hearing and walking have the same level of disability. Both persons would be classified as “disabled”. According to this assumption, a disability rate is estimated by simply counting the percentage of persons fulfilling a certain criterion, like the number of persons with important difficulties in at least two functioning domains.
Our psychometric analyses do not support this assumption. The item threshold for important difficulties in communicating is higher than for important difficulties in hearing, cognition and washing, which are, in turn, higher than those for important difficulties in seeing or walking. This means that persons with important difficulties in communicating are expected to have a severe level of disability (and difficulties in other domains), while those with important difficulties in hearing, cognition and washing, but not in communicating, are expected to have moderate levels of disability, and those with important difficulties just in seeing and/or walking are expected to have rather mild levels of disability. Wrongly assuming that important difficulties in at least two functioning domains is sufficient to decide that the level of disability a person experiences is severe might be one reason why the screener approach generated a disability rate of over 32%. In fact, this rate classifies 30% of persons with moderate disabilities erroneously as severely disabled.
Countries applying this rather easy and straightforward screener approach must be aware of the high risk of generating an overestimated, yet imprecise, rate of disability and of its consequences for policy making. The example of Brazil is instructive. In its 2010 census, Brazil included the WG-6 for the first time with the result that an impressive disability rate of 23.9% was estimated, i.e.
, 45.6 million people were classified as disabled [24
]. In the 2000 census, the disability rate had been 14.5%. In 2010, the highest prevalence of disability was seen, with 18.8% of the population claiming vision difficulties, probably because a large proportion of the population in Brazil had no access to an essential assistive device: glasses. As we demonstrate in the present work, however, persons with important difficulties in seeing are expected to have mild to moderate levels of disability, and in the case of Brazil, many might require only a simple public health intervention, such as the provision of glasses. The appropriateness of such high and not further differentiated disability rates can, therefore, be questioned in light of its consequences for intervention planning, policy making and the allocation of health and social resources to meet the needs of persons with different levels of disability.
The approach recommended in the WRD to estimate a disability rate is fundamentally different, as it uses questions of functioning domains to first create a scale of disability with metrical properties and then combines the average disability score of persons with disabling health conditions and the average disability score of persons with important problems in eight functioning domains to define the rate of persons with severe levels of disability. In other words, the WRD approach acknowledges that people with important difficulties in different functioning domains may have significantly different levels of disability associated with specific levels of performance and specific needs regarding interventions. The WRD approach also assumes that disability is a universal phenomenon characterized by a continuum ranging from low to high disability levels. As a consequence, disability must be measured by creating a scale with metric properties that integrates information about functioning domains and takes into account their differences. Our data show, for instance, that persons with important vision problems (but no problems in hearing, washing and communicating) are expected to have mild to moderate disability levels. As a consequence, these people would not be included in the group experiencing severe disabilities, as was the case in Brazil, and more precisely targeted interventions could be developed.
The WRD approach is initially more challenging, as it involves more time and personnel resources to recruit a general sample, to run a comprehensive disability survey like the MDS and to perform elaborate statistical analyses; but the approach pays off in terms of providing precise information about the percentage of the population experiencing significant disability and in terms of its usefulness in monitoring the implementation of the CRPD. By using a general population sample, this approach goes beyond defining disability in terms of having or not having a disability to that of providing countries with differentiated information about the level of performance and the specific needs of the population experiencing mild, moderate and severe levels of disability. Finally, it also provides a picture of the situation of persons without a disability, which can be set as baseline data providing an unbiased picture of the extent to which the environment in a country hinders or facilitates a person, even in the absence of a health condition.
This study should be understood in light of its limitations. Data used in the present work were obtained in a pilot study including a non-representative, convenience sample. Consequently, the group of persons without a disability was very small. Moreover, the criteria used as the disability screener are some of the many possible. Different criteria would have led to different disability rates. We, however, selected criteria that have often been used and that reflect the common practice in the field.