The Disablement Score: An Intersubjective Severity Scale of the Social Exclusion of Disabled People

Abstract

If a disability is understood as a type of social exclusion, its severity can be gauged from the social aspect. Such measurement is necessary to explore the intersubjective structure of social exclusion associated with bodily functions and structures. This paper presents a sociological and statistical method to rate the severity of a disability as social exclusion. The method is modeled on the rating procedure of occupational prestige. According to this technique, people subjectively rate severity by answering a questionnaire. The ratings are converted into a score (the “disablement score”). The method is applied in a preliminary web survey. The reliability of the scale is examined. People evaluate various conditions very differently, with physical conditions with functional limitations rated as severe and disfigurements as mild. Although the result does not necessarily agree with the objective circumstances, it is meaningful in that it reflects people’s reactions and attitudes toward disabilities.

1. Introduction

How can we assess the gravity of different types of disabilities? For instance, which disability is more severe (or roughly equally severe): deafness or blindness, the inability to walk or use one’s hands, or depression or social phobias? This differs from comparisons in a disability type (e.g., blindness and low vision), and the answer is not self-evident. Reliable methodologies are yet to be established. Some evaluations appear immediately feasible, for example, those based on the type and amount of basic activities hindered. However, we do not know the weight and relevance of each activity. Already at this point, we must introduce a social viewpoint.

This paper presents a sociological approach to gauging the severity of different disability types. Two ideas are adopted from the discipline. First, disability severity is understood as the gravity of social exclusion. When a disability accompanies extreme exclusion, it is considered severe. In contrast, when a disability concurs with mild exclusion, it is regarded as mild. In fact, a severe/mild disability is a type of severe/mild exclusion. By gauging disability severity, we disclose part of the structure of social exclusion, which is a valuable contribution to both disability studies and sociology.

Second, the methodology developed in this paper focuses on a somewhat common tendency in people’s subjective evaluation. Doctors and medical researchers are not alone in judging the severity of a disability. Average citizens in society also make such evaluations. Possibly, these subjective judgments have a structure; that is, we may be able to determine an overall pattern of people’s subjective evaluation of the gravity of various disabilities. If so, this social fact [1] awaits sociological investigation. Although it may or may not be consistent with the “objective” circumstances concerning disability and exclusion, the pattern itself is part of the social reality that guides us in our social life. This can be a clue to solving the problem of how to determine severity.

Section 2 provides a review of the way the original social model of disability, which also defines disability as a type of social exclusion, slighted the measurement of severity, and considers the theoretical underpinnings of measuring severity. Section 3 refers to various attempts to assess severity and relevant research. In Section 4, the strengths and weaknesses of objective and subjective approaches to the sociological measurement of severity are discussed and a severity scale called the “disablement score” presented. Finally, the method is applied in a preliminary web survey and the reliability of the scale examined in Section 5.

2. The Original Social Model of Disability and Severity

As known, the original form of the social model of disability was proposed by the Union of the Physically Impaired against Segregation (UPIAS), a disability rights organization in the United Kingdom. UPIAS distinguished between impairment and disability. For UPIAS, impairment meant “lacking part of or all of a limb, or having a defective limb, organ, or mechanism of the body,” whereas disability was a “disadvantage” caused by contemporary society, which “excludes” disabled people [2] (p. 14). From the standpoint of UPIAS, a disability can be understood as a type of social exclusion¹. If so, do severe exclusion/disability and mild exclusion/disability exist? Regarding the degree of exclusion (which equals the severity of a disability), UPIAS argued:

It is clear that our social organisation does not discriminate equally against all physical impairments and hence there arises the appearance of degrees of exclusion (degrees of disability). … Nevertheless, it is the same society which disables people whatever their type, or degree of physical impairment, and therefore there is a single cause within the organisation of society that is responsible for the creation of the disability of physically impaired people [2].

(pp. 14f.)

UPIAS maintained that differences in the degree of exclusion (which equals the severity of a disability) are not that important; rather, they believed that a single cause of disabilities should be emphasized. Likely, this was linked to the organization’s minority group approach. To gather disabled people into a consolidated minority, it must have been inconvenient to pay attention to different levels of exclusion.

However, in contemporary sociology, it is wrong to ignore the differences in degree of social exclusion². Niklas Luhmann [5] called the gravest aspect of social exclusion “exclusion from the whole society,” which indicates being excluded for a long period or from multiple areas of society (function systems) including law, politics, and the economy. By this, Luhmann distinguishes between severe and mild exclusion.

In disability studies, Ryoji Hoshika [6] refers to a similar situation as the “concentration of disadvantages,” in other words, experiencing disadvantages in almost all aspects or periods of social life. For him, like UPIAS, disability is a social disadvantage; however, severe disabilities, or concentrated disadvantages, can be distinguished from milder disabilities. Hoshika claims that it is justifiable to assign higher priority in policies to the concentrated disadvantages experienced by disabled people. The review above denies the line of reasoning proposed by UPIAS about the degree of disability³. Even when a disability is understood as a type of social exclusion, we can differentiate between more severe and milder disabilities. Since grave disabilities require intensive action, identifying them and gauging the severity of disabilities is a major task that sociology is expected to fulfill.

Therefore, we can gauge disability severity by measuring social exclusion⁴, rather than measuring the inherent properties of impairments, which cannot easily cross disability types. Some research attempting to quantify disability issues paid attention to the connection between social exclusion and disability, as examined in the next section. Before then, however, we must define the word “disability.” Without doing so, we cannot grasp the severity of what is measured. Although the severity scale presented here is also reasonable within the frameworks of the social model and the International Classification of Functioning, Disability and Health (ICF), this paper adopts a different definition. Here, disability is viewed as “social exclusion attributed to a link between fragmented body information and social treatment,” while an impairment is the “fragmented body information” constituting a disability [8]. “Body information” means an external reference to the body inside a social system, and “fragmented body information” is body information that refers to a limited aspect of the body (e.g., a body function or structure), rather than the body as a whole [8]. According to this definition, a disability is a type of social exclusion explained with reference to Luhmann’s theory. Consequently, assessing severity is directly linked to gauging social exclusion. This definition also incorporates the body (or more precisely, the social body), thus allowing one to measure the gravity of disabilities for each type of bodily condition (in this definition, types of disability and impairment can be similarly used). This definition has an advantage in that it does not determine impairment in an a priori way. Impairment does not mean an inherently inferior, imperfect, or deviant body, but a condition subject to social exclusion. Of course, social exclusion can be operationalized in more than one way. However, this definition keeps us aware of the social context in which social exclusion and disability are operationalized. For example, there are work and educational disabilities, which are associated with exclusion from work and education respectively. According to the definition above, bodily conditions such as the inability to see, hear, or walk can acquire different meanings in diverse social contexts, which is plausible. In this sense, that different operationalization is possible is not necessarily a defect.

Thus, this paper evaluates disability severity based on the degree of social exclusion, which constitutes a disability. Each type of impairment or functional limitation can be assessed. As mentioned, some quantitative studies on disability issues also refer to participation in social life. The next section provides a brief overview and explains the need for a new scale.

3. Quantitative Disability Research

3.1. Disability Statistics

Disability statistics do not necessarily assess the gravity of each type of disability; that statistics can determine severity is a new idea. However, since disability statistics constitute an important background to this paper, they are briefly discussed below.

Article 31 of the Convention on the Rights of Persons with Disabilities (CRPD, A/RES/61/106) requires the state parties to collect statistics as a basis for the implementation of the convention. At the time the CRPD was adopted in 2006, notable developments in disability statistics were emerging. Regarding the methodology, the Washington Group on Disability Statistics [9,10] and Budapest Initiative Task Force on Measurement of Health Status [11] developed survey questions pertaining to functional limitations. Among them, the Washington Group Short Set of Disability Questions [9] consists of six questions on bodily functions, namely vision, hearing, mobility, cognition, self-care, and communication. For instance, the question about vision asks the respondent: “Do you have difficulty seeing, even if wearing glasses?” These question sets are intended to identify disabled people in censuses and other surveys.

The World Report on Disability [12] reviewed international and national disability surveys. Besides the global prevalence rate of disabilities (15.6% to 19.4%), the report examines the socioeconomic status of disabled people, highlighting that disabled people of working age tend to be unemployed or work part-time when employed. The report refers to the 2009 OECD survey [13], which notes that working-aged disabled people are twice as likely to be unemployed than able-bodied people. Although it is indicated that they tend to be poorer than people who are able-bodied, this is not conclusive and needs further study.

The United States Survey of Income and Program Participation (SIPP) is an example of a national survey that contains information on disability and socioeconomic status. According to the 2008 SIPP [14], only 27.5% (estimated 5.57 million of 20.29 million people) of people with “severe” disabilities aged 21 to 64 years were employed, while 79.1% (estimated 116.88 million of 147.82 million people) of people with no disability in this age category were employed. The report also shows the employment rate and median family income by disability type, which can be used to measure disability severity. Although the SIPP microdata can be analyzed further, the abovementioned ratios cannot be utilized as they are because demographics and socioeconomic status are not controlled for. Therefore, a multivariate analysis is necessary, which manifests in the objective approach described below.

Nongovernmental disability statistics also exist. For example, Scope [15], an Australian nonprofit organization, conducted a survey on the social inclusion of disabled people. Conducted in 2011, the survey was conducted on 430 adults with disabilities in Victoria and incorporated subjective measures. Respondents rated the extent to which they viewed themselves as included in society. The highest possible response was 10. The items rated included the need for social participation (rated at 5.2 on average), satisfaction with relationships (6.2), and access to services (6.2). Although this is a subjective approach, it differs from the methodology developed in this paper. In the Scope survey, disabled people evaluated their own situation. However, people can also rate others’ circumstances concerning inclusion/exclusion, and these circumstances do not need to be real or specific cases. This leads to a new way of measuring disability, referred to in this paper as the “disablement score.”

3.2. Existing Severity Scales

Next, medical attempts to quantify the severity of disabilities are examined. The disability rating index [16] is self-administered and consists of 12 items including climbing stairs, running, and lifting heavy objects. For each scale, respondents are asked to indicate the degree of difficulty between 0 and 100. The mean provides the index, which expresses self-reported severity. Although this index includes work-related items, it focuses on activities of daily living and movement. As such, the index cannot be applied to different types of impairments (e.g., mental disorders). The functional independence measure [17] has the same limitation.

Unlike the scales described above, the discretized analog disability scale (DISS) formulated by Sheehan et al. [18] centers on social participation. The DISS consists of “work,” “social life,” “family life,” and two “perceived stress scales.” For instance, the question about work is: “The symptoms have disrupted your work” [18].

The options are 0 through 10 (the options for one perceived stress scale range from 0–100%, divided into percentages of 10%). Points can be self-administered or researcher-administered.

The DISS and the scale constructed here differ in the pattern of attribution. Attribution indicates an observation concerning what caused the situation in question. In this context, we focus on the attribution of social exclusion. Attribution does not mean finding an objective causal relationship, but depends on the observation and observer. The DISS pertains to two types of attribution of social exclusion in accordance with its use: the respondent’s individual attribution when self-administered, and the researcher’s individual attribution when researcher-administered. In a researcher-administered survey, doctors or other staff on the research team examine each existing patient case and attribute their social difficulties to their symptoms. This method is problematic in social statistics, because while the researchers influence the original ratings, they are not randomly selected. In a self-administered survey, the DISS presupposes respondents’ observation that their symptoms disrupted their own work, social life, or family life. Both the observer and observed are the respondents, who are selected through random sampling. However, this attribution may cause a systematic bias; social difficulties such as unemployment may encourage respondents to ascribe the hardship they experience to the body [19].

Thus, the attribution the DISS is based on is not completely appropriate in social statistics. However, other types of attribution can be used in social surveys: the researcher’s generalized attribution and respondents’ generalized attribution. The technique presented in this paper is based on the latter. In the next section, the construction of the scale is detailed.

4. Construction of the Disablement Score

Measuring the severity of disabilities corresponds to ordering disabilities from severe to mild. A similar ranking in social statistics is evident in studies on social stratification. A crucial component of social strata is assumed an occupational hierarchy, which can be determined through at least two methods [20] (p. 438). One is to determine the representative values (e.g., the mean and median) of each occupation’s income and academic qualifications (the objective method). The other is to aggregate people’s evaluations of jobs (the subjective method). Whereas the former entails a researcher’s direct observation of occupational status, the latter is a researcher’s observation of people’s observations [21] (p. 88). Although the objective method seems superior, it is not self-evident whether people rank occupations based on income or required academic qualifications. People do not necessarily view the occupational hierarchy obtained through the objective method as realistic [22] (p. 17). In addition, calculating the mean income and educational level for each vocation is not enough, because incumbents’ demographic traits affect them [20] (p. 439). Thus, controlling for demographic variables is needed. While it is possible to control for some demographic variables, it is not clear whether perfect controlling is possible. This requires a census-scale sample, and even in censuses, controlling depends on the model adopted by the researcher. Usually, only a fixed form of effects (e.g., linear, polynomial, etc.) and interactions (e.g., linear-by-linear) are considered, and not all possible interactions are entered into the formula (saturation models are not considered worthwhile). In addition, some jobs may carry too much bias in terms of incumbents’ demographic characteristics (e.g., gender, age, etc.) to control. In this sense, the subjective method, based on people’s occupational evaluations, is also meaningful. One example is the occupational prestige score, which is an aggregate of people’s observations. As a consensual scale, some authors characterize its nature as intersubjective [22] (p. 17) [23]. In the General Social Survey conducted by the National Opinion Research Center in the United States, respondents were asked to rate the social standing of each occupation on a scale ranging from 1 to 9 using cards and boxes. The mean rating for each vocation is the occupational prestige [24] (p. 3). In general, the ratings can be converted to a 100-point score at regular intervals; for instance, a 5-point scale corresponds to 0, 25, 50, 75, and 100 [20] (p. 445). The score varies little by time and area, and the prestige scores of different subgroups with different features are highly correlated [25]. Based on these characteristics, the occupational prestige score has been used as a stable marker of stratification structure.

In the context of disabilities, the researcher’s generalized attribution corresponds to the objective method of determining occupational status, and the respondent’s generalized attribution correlates with the subjective method. They can be implemented as follows. The objective measurement of severity gauges the extent of exclusion experienced by people associated with different bodily conditions. For instance, if we choose unemployment as an indicator of exclusion, we can measure the severity of a visual disability by calculating and comparing the employment rates of people who have difficulty seeing and those who can see.

In contrast, the subjective method asks people the extent to which different bodily conditions lead to social exclusion. The questions are generalized, as they probe the extent to which a person in general—not a specific individual—experiences disadvantages in social life for each bodily condition. Participants choose the extent of anticipated exclusion from the options, after which the responses are aggregated.

These methods have the following strengths and weaknesses. First, the objective method has the same problem with being realistic as the occupational prestige score. For instance, the importance of employment status for each respondent is not clear. On the other hand, the subjective method can consider the weights respondents attach to indicators of exclusion. Nevertheless, this point is not as critical for now. In social statistics, it is significant to quantify the social exclusion of disabled people based on factors such as employment rate, income, and academic qualifications. Consequently, the objective method should be developed as an important approach in sociological disability statistics.

However, difficulties are found in sampling and flexibility. A census scale sample may be needed to determine a significant relationship between exclusion and various bodily conditions, because a disability is expected to be a rare phenomenon. The prevalence of some conditions may be well below 1%, and even smaller among the working-age population. Consequently, the sample size may be too large to be available for independent researchers, requiring a nation-wide government survey such as a census. However, only a limited number of body-related questions can be incorporated into a census or existing survey. Conditions cannot be added to or removed from a survey at will. Health-specific government surveys such as the National Health Interview Survey of the United States [26] can deal with many conditions; however, the types of conditions are restricted to those traditionally treated by medicine as impairments. In addition, information on each item is limited, since thorough physical checkups are not feasible.

In contrast, the subjective (especially intersubjective) method has the advantage of being able to derive a conclusion from a normal-sized sample survey. However rare a bodily condition, people can anticipate its severity as a disability, and several hundred samples are probably adequate to score severity. This can be conducted as a separate survey by an independent researcher. Subsequently, the type and number of body-related questions can be more flexible, which is useful in exploring social meanings concerning bodily conditions, especially those not traditionally treated as impairments.

Another advantage concerns bias, which remains even when the required sample size is available. People who have difficulty in communication may be excluded from a social survey, which is a type of communication. These people may have difficulty reading, listening to, understanding, and answering the questions, or noticing the survey, rendering the sample biased. However, the bias caused by the subjective method may be milder. First, this approach deals with the mean ratings of the entire population, most of which is not disabled. As a result, the bias can be alleviated through aggregation when employing the subjective approach. Second, the bias can make more sense in terms of the subjective method. Even if respondents tend to be able-bodied, the subjective method, the main target of which is able-bodied people, is still suitable. The outcome indicates the disability phenomenon from people’s perspective, especially people who may exclude. Therefore, we can interpret the results even when the bias occurs.

Unfortunately, there are some disadvantages when using the subjective approach. In particular, respondents may have limited or no knowledge of the bodily conditions listed. These types of disability cannot be surveyed using this method. In addition, even when respondents have slight knowledge of a body condition, it may differ from the situation indicated by the objective method. Moreover, wording will considerably affect responses. In the final analysis, both the subjective and objective approaches are necessary, as they play a complementary role.

As such, it is possible to construct a subjective scale of disability severity based on respondents’ generalized attributions by asking the following question, which is modeled on the occupational prestige score.

Below is a list of physical and mental conditions. Some might negatively affect one’s social life such as work, school life, marriage, housework, and childcare. In your view, to what extent does each condition impact one’s social life?

(Supplementary Materials Questionnaire S1)

In the survey below, respondents were asked to choose among options ranging from 1 (“not adverse at all”) through 6 (“extremely adverse”). Scores for the options were assigned at 20-point intervals, with 100 indicating “extremely adverse” and 0 indicating “not adverse at all.” Then, the average score for each type of bodily condition was calculated. This is referred to as the “disablement score” for the respective bodily condition. The term disablement focuses on social exclusion, and is considered suitable for differentiating the disablement score from medical indices.

Rather than using the question above, we could ask directly whether each bodily condition, if considered an impairment, is severe or mild. However, this approach was not adopted, because the answer is difficult to interpret. The latter question asks not only about severity, but also the traditionality of a disability, i.e., the extent to which the condition has traditionally been treated as disability. These different dimensions require distinction. In contrast, the question adopted in this paper is restricted to severity and can be applied to “nontraditional” [27] (p. 178) disabilities, meaning that new conditions can be located on the scale.

Thus, the list of rated bodily conditions should include nontraditional and traditional disabilities to fully exploit this technique. Referring to the Extended Question Set of the Washington Group [10] and list by the Job Accommodation Network [28], and by adding conditions, the author formulated the following list, which can be further expanded:

• Inability to see
• Blind in one eye
• Inability to hear
• Inability to see and hear
• Inability to smell
• Inability to taste
• Inability to distinguish colors
• Inability to walk
• Using a wheelchair to move
• Inability to move one’s hands
• Inability to speak
• Inability to speak fluently and repeating words
• Inability to understand words
• Inability to memorize something new
• Inability to stop drinking alcohol
• Inability to stop smoking
• Feeling sleepy in the daytime
• Feeling depressed and like doing nothing
• Hallucinating or having delusions
• Afraid of meeting others
• Extremely small in size
• Very overweight
• Very thin
• Having a birthmark on the face
• Having no hair
• Always feeling pain somewhere in the body
• Inability to sit still due to lower-back pain
• Getting tired extremely easily
• Total inability to understand others’ feelings
• Inability to stay still
• Inability to read letters, though able to see them

This list consists of conditions as extreme as possible. When employing the subjective method, many respondents find it challenging to imagine partial functioning, and including it causes the problem of cut-off points. Therefore, asking about total inability when possible is preferable for rating the disablement score.

5. Reliability and Structure of the Disablement Score: A Preliminary Survey

5.1. Materials and Methods

A preliminary survey of the disablement score was commissioned to a Japanese research agent (NTTCom Online Marketing Solutions) and conducted on the Internet from 26 May to 30 May 2017. The survey was permitted by the Research Ethics Board of Waseda University in Tokyo. Of the two questionnaires used, Form A included the question above, and is dealt with below. In total, 260 people from the registered panel responded to Form A, of which 224 responses were valid. Although not randomly sampled, it is worthwhile as an introductory examination of the scale. The survey was entitled “Attitude Survey of Social Participation and the Body”. Terms such as “disability” were avoided in the title and description to address nontraditional and traditional “impairments” and focus on the severity of exclusion.

5.2. Results

Table 1. Demographics.

	Mean ± Standard Deviation or Frequencies
Sex	Male: 143 (63.8%), Female: 81 (36.2%)
Age	53.4 ± 12.5
Experienced a serious injury/illness	Yes: 63 (28.1%), No: 161 (71.9%)
Subjective health status	2.39 ± 1.04
(1 = Good, 5 = Poor)
Education	Elementary: 1 (0.446%), Secondary: 73 (32.6%), Higher: 144 (64.3%), DK: 6 (2.68%)
Household income before taxation	5.82 ± 3.47 (from the frequency table)
last year (million yen)	DK: 17 (7.59%), Answer Denied: 23 (10.3%)

provides the demographic items. The survey attracted more men than women, possibly because it focused on social participation. As discussed later, however, men and women share a common rating pattern, which justifies aggregating both genders. The reason one person is an elementary school graduate is that the person finished school before educational reform after World War II.

Table 2. Disablement Scores.

	Disablement Score	Standard Deviation
		Single	Mean
Unable to see and hear	90.8	24.6	1.64
Unable to see	88.7	24.1	1.61
Unable to hear	83.5	25.3	1.69
Unable to move one’s hands	81.1	26.6	1.77
Unable to understand words	80.9	26.3	1.76
Unable to walk	80.0	28.2	1.89
Unable to speak	79.1	27.9	1.86
Unable to memorize something new	73.7	26.1	1.74
Using a wheelchair to move	72.2	28.9	1.93
Hallucinating or having delusions	71.2	29.6	1.98
Unable to read letters, though able to see them	68.1	28.3	1.89
Unable to distinguish colors	67.2	27.7	1.85
Afraid of meeting others	67.1	29.4	1.96
Blind in one eye	66.6	26.1	1.74
Unable to speak fluently and repeat words	63.1	27.6	1.85
Unable to smell	60.8	29.3	1.96
Unable to taste	60.6	29.6	1.98
Totally unable to understand other’s feelings	60.2	29.4	1.96
Unable to stay still	58.8	27.8	1.86
Unable to keep sitting due to lower-back pain	58.3	29.0	1.93
Always feeling pain somewhere in the body	57.7	29.1	1.94
Feeling depressed and feeling like doing nothing	57.0	27.4	1.83
Getting tired extremely easily	56.6	27.7	1.85
Feeling sleepy in the daytime	51.1	26.6	1.77
Unable to stop drinking alcohol	51.0	28.2	1.88
Unable to stop smoking	46.6	29.3	1.96
Very overweight	40.2	28.9	1.93
Extremely small in size	38.0	27.7	1.85
Having a birthmark on the face	37.9	28.2	1.88
Very thin	33.5	26.9	1.80
Having no hair	31.4	28.3	1.89

Note: N = 224. Respondents ratings (1–6) are converted into 0–100 range; the mean values are disablement scores in the second column, and the standard deviations of the single responses (0–100) are in the third column (σ). The fourth column is the estimated standard deviation of each disablement score ($\mathsf{\sigma }/\sqrt{\mathrm{N}}$).

provides the disablement scores in descending order. Before considering each condition, respondents were asked to select “Not adverse at all” (1) to avoid reverse responses in which options 1 and 6 were confused. All responses of respondents who selected options other than 1 in this question were considered invalid and removed from the analysis. Among the valid answers, 10 rated blindness in one eye higher than blindness, blindness higher than deaf-blindness, or deafness higher than deaf-blindness. In addition, 14 non-discriminant responses selected the same option (e.g., 6) throughout. Although these responses are doubtful, they were included in the analysis, unless otherwise stated, to avoid imposing the researcher’s assumption.

Before interpreting the disablement scores, the reliability of the scale is examined. However, the scale of the disablement score, which follows occupational prestige, differs from ordinary psychological scales. First, of interest is the average points of each item (a bodily condition) among different individuals, rather than the mean (or the total) score of each individual among different items. Second, only one aspect is gauged for each item (bodily condition), although more aspects can be measured. Third, hundreds or thousands of raters can be randomly sampled, rather than a small number of privileged raters. Because of these differences, we must develop a way to calculate the reliability of the disablement score and other scales like occupational prestige.

One way is to apply Cronbach’s alpha [29] to the transposed data matrix with bodily conditions in the rows and respondents (=raters) in the columns. Cronbach’s alpha is a well-known indicator of the reliability of data, which ranges from 0 to 1. Usually, the rows bear the individual subjects who are evaluated and the columns contain the items of evaluation. The measure for which reliability is derived is the total (or average) scores marked by each individual for the rating items. Cronbach [29] notes that if the rating items are random samples from an item population, and if two sets of items are sampled, the expected value of the correlation coefficient between the two sets of aggregate scores marked by individuals for the two sets of rating items equals Cronbach’s alpha. This characteristic can be applied to the disablement score: If we derive two sets of disablement scores for a list of conditions from two groups of randomly sampled respondents, the expected value of the correlation coefficient between the two sets of scores equals Cronbach’s alpha calculated from the conditions × respondents data matrix. This value reflects the stability of the tendency of the disablement score. The reliability thus calculated is 0.99 (the 95% confidence interval: (0.99, 0.99)).⁵

Another way to evaluate the reliability of the disablement score and other scales like the prestige score is to employ the intra-class correlation coefficient (ICC) [31], again based on the transposed data matrix. If we suppose each single score for a specific condition rated by a particular rater is composed of the true value of the disablement score for the condition, rater’s effect, and error, we can estimate the variance of each term. The ICC is the proportion of the estimated variance of the true value to the total estimated variance, which ranges from 0 to 1. The ICC of the single score composing the disablement score is 0.30 excluding the non-discriminant cases (the 95% confidence interval: (0.22, 0.44). The ICC of the mean value among the 224 raters of the disablement score is 0.99 (the 95% confidence interval: (0.98, 0.99), which is derived from the single value ICC using the Spearman-Brown formula.

On the other hand, the validity of the scale is currently difficult to examine, because no established and statistically confirmed list pertaining to disability severity exists. Consequently, no external criterion with which to compare the disablement score exists. In the future, such a criterion could be developed through the objective approach, but further studies are needed.

In addition to this evaluation of reliability, the stability of the scale among demographic attributes is examined.

Table 3. Correlations of Disablement Scores between Subgroups and the Entire Sample.

		Correlation	N
Sex	Male	0.998	143
	Female	0.994	81
Age	<40	0.966	28
	[40, 50)	0.995	58
	[50, 60)	0.993	70
	[60, 70)	0.993	42
	≥70	0.971	26
Experienced a serious injury/illness	Yes	0.992	63
	No	0.999	161
Subjective health status	1 (Good)	0.983	52
	2	0.997	67
	3	0.997	77
	4, 5	0.970	28
Education	Secondary	0.992	73
	Higher	0.998	144
Household income before taxation last year (million yen)	<2	0.976	19
	[2, 4)	0.991	49
	[4, 6)	0.993	39
	[6, 8)	0.986	30
	[8, 10)	0.985	27
	≥10	0.956	20

Note: Some categories are merged because of the small size.

shows the correlation coefficients between the scores of sub-groups and the entire sample. Some categories were integrated because of their small size. For the attributes examined here, the correlations are more than 0.95.

5.3. Discussion

The reliability of the disablement score is remarkably high, suggesting satisfactory reliability. In addition, the scale proved to be stable, meaning that the tendency of the disablement score hardly varies among different sub-groups. Note that reliability depends on the sample size of respondents and number and type of bodily conditions selected. Confining the list of bodily conditions to those with a similar disability severity may decrease reliability. However, based on the variety of conditions adopted, the analysis shows that the average of the slightly more than 200 respondents yields a considerably reliable rating scale.

Because of this characteristic, we can now interpret the disablement score. In general, sensory and physical conditions with functional limitations rank high, for example, deaf-blindness, blindness, deafness, leg paralysis, and hand paralysis. These conditions are traditionally treated as severe and typical physical disabilities. As such, the result statistically reconfirms and quantifies the conventional perception of disability severity.

Of these conditions, blindness, deafness, leg paralysis, and hand paralysis are similarly rated (the 80-point range). Although this appears trivial, it was an intuitive idea now corroborated by statistical evidence. Interestingly, the disablement score for wheelchair use is somewhat lower than that for leg paralysis. This may be natural, because in addition to the positive wording, it reminds us of the possibility of support and barrier-free. Although the disablement score is influenced by the aspect of a condition focused on, this does not invalidate the scale; rather, it closely reflects the sociological phenomenon around the condition.

The highest group of conditions includes an intellectual condition (i.e., inability to understand words). The disablement scores for mental or psychiatric conditions (e.g., delusion, social phobia, and depression) are slightly lower overall, although delusion approaches the highest group. This does not imply that these conditions are milder, since at least the items for delusion and social phobia do not specify the extreme status. Specifically, the type and frequency of hallucinations and delusions are important, but it is difficult to include a brief and non-leading expression for these aspects. Furthermore, there may be a longer distance between medical (psychiatric) diagnoses (e.g., schizophrenia) and their nonprofessional expression than for physical conditions; the former may sound like a sign of otherness, whereas the latter can be dismissed as an extension of normal human feelings. The questionnaire avoids medical diagnoses and adopts nonprofessional expressions to ensure ordinary citizens’ better understanding. Further study is needed on the extent to which the scale grasps the gravity of the exclusion accompanied by mental conditions.

Disfigurement (e.g., a birthmark on the face, loss of hair, being overweight, and dwarfism) is rated even lower, suggesting that people attach greater importance to functional limitations than a different appearance and the negative attitude elicited when attributing social disadvantages. This interpretation is also supported by the fact that major traditional physical impairments and wide-ranging sensory and physical conditions with functional limitations are considered disadvantageous in social life. For example, respondents evaluate smell and taste disorders as moderately disabling, implying that people discriminate between a class of conditions (e.g., among sensory conditions).

Therefore, the disablement score is a scale of perceived dis-ability in a narrow etymological sense, and dis-order and dis-figurement are possibly somewhat marginalized. This may be a limitation of the scale. However, this reflects an aspect of the social structure of meaning. People associate social disadvantages more closely with the functional limitations of the body. This does not mean that people who have bodily conditions without functional limitations do not experience severe exclusion. As mentioned, the subjective and objective approaches do not always concur.

6. Conclusions

This paper developed an intersubjective rating scale of disability as social exclusion called the disablement score. By this measure, we can observe people’s observations of disabilities. People have a commonsense view of bodily conditions and social exclusion; as such, the disablement score illuminates the structure of people’s recognition. By this scale, we can explore the meaning people attach to the body, rather than the material body itself. Consequently, the disablement score is entirely a sociological matter. This paper is an attempt to apply a method of sociological statistics to measure disabilities. Further development in the field of sociological disability statistics is hoped for.

The preparatory data suggests a strong and stable structure of perceived disability severity. People widely agree on how disabling bodily conditions are. This structure remains even when barrier-free and social support have developed to some extent. Although the identified structure does not necessarily agree with the actual severity of exclusion, it is still important, as perceived severity may be an indicator of people’s negative attitudes and reactions when they know a colleague or a partner has a bodily condition.

Since this survey was a closed one, a randomly sampled survey is needed next to confirm the results. In addition, a statistical inquiry based on the objective method is also needed for comparison with the survey in this study. Furthermore, international comparison of the disablement score is of great interest. These topics cannot be investigated here and await further research.