The Problematics of Assessing Trans Identity in Survey Research: A Modest Proposal for Improving Question Design

Abstract

One of the central issues facing the trans community today is not only to be counted, but also how to be properly counted. If and how trans people are counted has a huge impact on what we know, or what we think we know, about the trans community. When trans people are not counted, we know nothing, but when trans people are counted incorrectly, the results can be even worse. The question addressed in this paper, therefore, is how to develop a question(s) that will more accurately account for the trans population on national surveys. By drawing on cognitive interviews testing a gender identity and sexual identity question for a national official health survey, an argument is made for an improved method of understanding trans measurement on surveys.

1. The Problematics of Assessing Trans Identity in Survey Research: A Modest Proposal for Improving Question Design

The terminology used to capture what is commonly referred to as the transgender community is still under construction and negotiation [1]. Following Ryan [2], the term “trans” will be used throughout this paper “as an imperfect shorthand to refer to individuals whose current gender identity does not match the social expectations of their medically assigned sex at birth. This group could include, but is not limited to, those who identify as transgender, transsexual, genderqueer, third gender, or gender non-binary, among many other possible identities that would be seen as gender non-conforming.” Existing research has shown that the trans population is particularly vulnerable to social and legal exclusion, economic vulnerability, and violence. Significant further research is needed to improve our understanding of the full range of issues facing the trans community and this paper suggests one way in which that research might be improved.

One of the central issues facing the trans community today is not only to be counted, but also how to be properly counted. If and how trans people are counted has a huge impact on what we know, or what we think we know, about the trans community. When trans people are not counted, we know nothing, but when trans people are counted incorrectly, the results can be even worse. Many studies, mostly needs-assessment and behavioral risk surveys, have shown the negative social and health differences suffered by trans people with issues related to HIV/AIDS [3,4], self-harm and suicide [5,6], homelessness [7,8], and a variety of other negative health outcomes. Although many contest this data, including how it was collected, few contest the actual construction of the question used to collect the data. Thus, an improved understanding of question design could potentially reshape the ways in which we view current survey dependent knowledge of the trans community [9,10,11].

One of the principle problems facing those trying to capture an accurate picture of the trans community through surveys is that many trans people themselves do not want to be counted as such [12]. In many ways, doing so would defeat what many have tried to achieve. For many, being transgender is more of a process, a means to an end, than it is an end goal. It is not that they want to be trans but that they want to transition to another sex and/or gender (For a more elaborate discussion on this, see Roen 2002 [13]). That said, there are still arguably a bedrock of common issues faced by trans people as a “community” that warrant some kind of official count so that these issues can be more properly understood. The question then becomes—how can we develop a question(s) that will more accurately account for the trans population on national surveys? This paper will suggest an important first step toward answering that question.

Perhaps the most common situation where questions (or response options) related to trans identity appear on national surveys is in the context of either a gender or a sexual identity question. Although significant research has been done on the ways in which trans people make sense of their gender and/or sexual identity (see, for example, discussions in Stryker and Whittle, 2006 [14]), there are still research gaps on how trans people make sense of their gender and/or sexual identity on official surveys [11]. This is an important question, as data from official statistics has important political and policy consequences. In this paper, an examination will be made of cognitive interviews conducted with twenty-one trans-identified respondents to both a gender as well as a sexual identity question in order to get a better understanding of how respondents made sense of these questions, and particularly of how they made sense of their own status as a trans person on an official survey.

2. Current State of Trans Identity Measurement on Surveys

Previous research has primarily utilized two different methodologies to study the trans population: needs assessment studies conducted on a local and regional level and surveys conducted through non-probability sampling to target the national population, largely done using the internet.

A number of needs assessment surveys have been conducted in order to gain insights about health patterns among trans individuals. These studies tend to focus on accessible trans populations, such as sex workers or clinical samples. Relying on specific segments within the trans population to make inferences about the larger trans population can have a number of negative consequences, such as over-representing certain health conditions, particularly when tied to particular demographics [3]. While these needs assessment studies are helpful in understanding the respondents included, due to the sampling methods, we cannot use the results found in these types of studies to make inferences about the trans population as a whole or even of the trans population in that community.

Researchers have also turned to online surveys in order to learn about the trans population at large. These surveys have the advantage of capturing respondents who do not openly identify as trans; however, there is currently no method to randomly sample online, and thus researchers rely on gathering large samples in an attempt to compensate for this limitation. The largest of these surveys, The National Transgender Discrimination Survey (NTDS), interviewed 6456 respondents who identified as transgender or gender non-conforming, using a web survey that was augmented with paper questionnaires for difficult to reach populations [15]. Ultimately, the survey received responses from respondents in all fifty states in the United States of America plus the District of Columbia, Puerto Rico, Guam, and the US Virgin Islands. Survey respondents reported lower incomes and higher unemployment rates compared to the rates reported by the Bureau of Labor Statistics for the general United States population. Respondents were also more likely to be younger and to be white. Additionally, while a large number of studies have focused on trans individuals who are sex workers, only eleven percent of respondents to the NTDS reported ever having exchanged sexual services for financial compensation. The survey also found that respondent sexual identities varied greatly. Despite the large number of responses and the regional diversity of the responses, we cannot assume that these results are representative of the national trans population. In surveys of the United States population at large, we could compare the reported demographics of the survey to known population totals from the Census; however, in this case, there are no known population estimates for trans-identified people to use for comparison. Ultimately, while surveys like the NTDS take a large first step in conducting surveys of the trans community, its results are only representative of those who responded to the survey.

The needs of trans people are often not represented on official surveys largely because we do not yet have an accurate way to measure trans identity [11]. Typically, transgender respondents have been identified on surveys with three different approaches. The first approach is to ask two separate questions—one on birth sex and the other on current gender identity (as used by Rosser et al., 2007 [16]). An “inconsistency” between the two answers leads to a classification of the respondent as trans. Although this option is less likely to put off non-transgender respondents, it also suffers a number of drawbacks. It is an indirect way of assessing transgender status and, therefore, relies on analyst interpretation rather than respondent identification as a trans person. It is also often contested by large surveys who do not wish to add an additional question to what are, quite often, already lengthy assessment surveys.

The second approach is to simply ask directly if a respondent is trans or not (as used, for example, in the Massachusetts Behavioral Risk Factor Surveillance System, 2007 [17]). This approach has the benefit of being more direct and relying exclusively on respondent identification. A drawback of this approach, however, is that there are many individuals who researchers might classify as trans but who do not themselves identify as such. In other words, respondents who are born male and transition to female now consider themselves to be simply female rather than trans.

A third approach is to add a trans response option to an existing gender and/or sexual identity measure (as, for example, in research by Conron, Scout, and Austin, 2008 [18]). This approach has the benefit of not increasing survey burden with additional questions and also not asking respondents if they are transgender in a limited context. The addition of a response option for trans identity, especially when situated among other response options, has the benefit of allowing respondents to select this option within the context of other gender options.

3. Methods and Data Analysis

The first step to developing any good survey question is to understand how respondents interpret and comprehend the question. According to Tourangeau, Rips, and Rasinski’s model [19], respondents must comprehend the question, retrieve relevant memories, integrate all of the relevant memories or facts, and finally map this information onto the provided response options. Each of these tasks presents an opportunity for error. As a qualitative method of pre-testing survey questions, cognitive interviewing allows researchers to follow the steps taken by the respondent to arrive at their final answer. Additionally, cognitive interviewing allows researchers to note different interpretations of the question and response options across respondents [20]. Cognitive interviewing is currently the primary method of testing utilized by federal statistical agencies in the United States. Ultimately, cognitive interviewing provides rich narratives that can be used to better understand patterns of interpretation across respondents and demographic groups, which can ultimately be used to ensure that the survey question performs as intended [21].

Data for this analysis comes from 21 cognitive interviews conducted with respondents in the United States who self-identified as transgender, transsexual, or genderqueer. Although there is some debate as to the sample size necessary to obtain valid results from a cognitive interviewing study [22,23], there is relative consensus that the goal of cognitive interviewing is to saturate patterns of interpretation, not to make generalizable population estimates. Thus, the true measure of a successful cognitive interviewing study is one of pattern saturation rather than a question of the number of respondents.

The respondents in this study reflect a wide range of backgrounds. Eleven interviews were conducted in English, while ten were conducted in Spanish. All Spanish translations were conducted by the author and are shown in English below. Two respondents had an elementary school education, two had attended high school but did not obtain their diploma, four respondents had a high school diploma or General Education Diploma (GED), three had an Associate’s Degree, five had a Bachelor’s Degree, and one had a Master’s Degree. Respondents ranged in age between 21 and 51 years old, with the majority of respondents being in their thirties. Additionally, six respondents identified as White, four identified as Black or African-American, three respondents identified as multiracial, and the remainder identified as “some other race” (this occurred primarily with Spanish-speaking respondents as Hispanic and Latino were listed as ethnicities, not races, on the tested questionnaire).

The interviews were conducted as part of a larger project testing a revised sexual identity question for the National Health Interview Survey (NHIS). Interviewers from the Questionnaire Design Research Laboratory (QDRL) at the National Center for Health Statistics (NCHS) conducted the interviews in July and August of 2011 [24]. All interviews were conducted in a major metropolitan area in the United States. As is common with cognitive interviewing, respondents for this project were selected using purposive sampling. Respondents were recruited through email by a number of organizations serving the Lesbian, Gay, Bisexual, and Transgender (LGBT) communities as well as by word of mouth.

Respondents were scheduled for specific interview times (with the exception of a few “drop-ins”) and reported to a set location for their interview. Interviews lasted between 30 and 60 min, with the typical interview lasting from 45–60 min. While all interviews were conducted anonymously, respondents were asked to consent to the audio recording before the interview and again once the interview began. At the conclusion of the interview, all respondents were given $50 as remuneration. Interviewing for the project continued until theoretical saturation was reached—that is, interviewing was continued until no new patterns of interpretation were detected. The number of interviews required to achieve saturation can vary greatly; for example, a recent empirical study found that saturation was achieved in as few as twelve interviews [22]. For this project, a total of 139 cognitive interviews were conducted and 21 of the interviews were conducted with transgender, transsexual, or genderqueer-identified respondents before researchers felt confident that saturation had been reached.

The questionnaire for this project was administered using an audio-computer-assisted self-interview (ACASI) instrument, which has been shown to improve data quality in potentially sensitive questions such as sexual identity [25]. Respondents were asked to answer a number of demographic questions using the ACASI system without assistance from the interviewer.

At the conclusion of the questionnaire, respondents were asked each item and were then asked to explain their answer. Typical follow-up questions included, “Why did you answer in this way?”, “What do you mean by that?”, and “What does term X mean to you?”. If a respondent’s answer seemed vague or unclear, the interviewer asked questions such as: “Can you give an example to describe what you are talking about?” Specifically, for the sexual identity question, respondents were also asked how they typically referred to themselves and were also asked about other words that did not appear in the question. The culminating text from the interview related how respondents understood or interpreted each question and also outlined the types of experiences and behaviors respondents considered in providing an answer.

Interview data was analyzed using the constant comparative method as described by Ridolfo and Schoua-Glusberg [26]. The constant comparative method is an inductive method of analysis that relies upon systematic coding of interview responses along with analysis of the interview data to develop theories. Once the interview was completed, the interviewer’s notes along with the audio recording of the interview were entered into a qualitative data analysis program designed especially for the analysis of cognitive interviews. As data was entered into the program, the interviewer’s notes were reviewed by analysts to ascertain how the respondent interpreted the question and to determine what information the respondent used to explain why they answered the way they did. After all interviews were completed, the patterns that emerged during the initial round of analysis were refined and developed into categories and themes that were then applied to group the applicable interviews (for more on data analysis of cognitive interviews, see Miller, et al. [27]). As a final step, interviewers were compared by themes of race, education level, and language of the interview to determine whether there were any similarities or differences between these groups.

4. Understanding Trans Identity through Response Options

An examination of the patterns of interpretation of trans respondents to both a gender as well as a sexual identity question will be conducted in order to better understand how trans identities might be better captured on official surveys. A further examination of trans respondents’ responses to a gender identity question as well as to a sexual identity question individually will be conducted and then a comparison will be made between the two response sets to look for overlapping patterns. It should be noted that in both questions, trans-related identities appeared as a sub-option—that is, as a follow-up option—to one of the primary response options.

4.1. Transgender Identities Reflected through a Gender Identity Question

All respondents were asked the gender identity question below. This question appeared as the first question on the survey and was written with the goal of providing trans respondents a response option outside of the traditional dichotomous male and female response options.

English: Do you consider yourself to be… Male, Female, or It is more complicated (Go to 1a)?

Spanish: Usted se considera ser… Hombre, Mujer, o Es más complicado (Go to 1a)?

English Followup: [If it is complicated is selected] By answering it’s complicated, do you mean that…

Male, assigned female at birth

Female, assigned male at birth

Masculine, assigned female at birth

Feminine, assigned male at birth

Transgender or genderqueer, assigned female at birth

Transgender or genderqueer, assigned male at birth

Something else

I didn’t mean to choose this option

Spanish Followup: [If it is more complicated is selected] Cuando dice es más complicado, quiere decir que…

Hombre, al nacer asignado como mujer

Mujer, al nacer asignado como hombre

Masculino, al nacer asignado como mujer

Femenina, al nacer asignado como hombre

Transgénero o géneroqueer, al nacer asignado como hombre

Transgénero o géneroqueer, al nacer asignado como hombre

Algo diferente

No quise elegir esta opción

Responses from trans respondents varied greatly and ultimately reflected the diversity of their backgrounds.

Table 1. Responses to Gender Identity Question.

Responses	Frequency
Male	5
Female	9
It is more complicated	7
Transgender or genderqueer, assigned male (3)
Transgender or genderqueer, assigned female (2)
Masculine, assigned female (1)
Feminine, assigned male (1) Total	21

summarizes trans respondents’ responses to the tested gender question.

The differing responses to this question are likely a reflection of the fact that respondents were at differing points in the process of identifying as trans. Respondents who selected male or female tended to be further along in the transition and/or self-identity process, while those who had not yet begun or were at the very beginning of the process tended to gravitate more towards the ‘it is more complicated’ response option. One respondent who had completed their transition explained why they did not select ‘it is more complicated’ by saying, “I felt the wording to be… not a comfortable fit for me, the wording was… awkward is not necessarily what I would describe it as. I don’t find it to be complicated because I know what it is. Trying to articulate it is a different matter. I don’t want the perception be that it is more complicated.” Another respondent who completed his transition to a male explained that he liked seeing the third option (that is, something other than ‘male’ and ‘female’) available and even considered selecting this option but said, “But I chose male because whenever I fill out any paperwork and whenever I self-identify its male.”

One consistent finding among trans respondents is that they liked having a third option available when discussing their gender. However, they found the current wording to have a negative connotation. Twelve trans respondents indicated that they either didn’t like the wording of ‘it is more complicated’ or said that the option was “not for them”. Respondents stated that the ‘it is more complicated’ response option was for people who were still questioning their gender or people transitioning genders. Additionally, three respondents stated that they did not view their gender as being complicated and, therefore, they did not feel that this response option was right for them. One respondent said, “I don’t see it as being complicated, just different.” Another respondent asked, “Why is it complicated that I’m neither [male or female]?” While another respondent went so far as to say that he felt that the phrase “it is more complicated” made it sound like he “had issues” because of his trans status.

Additionally, trans respondents described their gender as being socially constructed. Interviews were coded as “gender as socially constructed” if the respondent mentioned their behavior, actions, appearance, clothing choices, or hobbies in explaining their gender. Overall, seventy percent of trans respondents described their gender as being socially constructed. In explaining what defines them as a female, one trans respondent said, “[Its] Not so much biological, but mental… In my mind I more associate with the female gender.” Responses similar to this were very common in interviews with trans respondents.

Another issue arose with the ‘it is more complicated’ follow-up question. Four respondents in Spanish interviews noted that they found the response options under the ‘it is more complicated’ follow-up question to be confusing. Many respondents noted that they were confused about the differences between the response options. This problem did not arise in English interviews, indicating that Spanish-speaking trans individuals might use different terminology than English-speaking trans individuals or that there are some problems in the translation of the terminology from English. The higher than average education levels of the English-speaking trans respondents might also explain why none of them found the response sub-options for the ‘it is more complicated’ follow-up to be confusing.

4.2. Transgender Identities Reflected through a Sexual Identity Question

All respondents were asked the following sexual identity question:

English: Do you think of yourself as:

[For men:] Gay

[For women:] Lesbian or gay

[For men:] Straight—that is, not gay

[For women:] Straight—that is, not lesbian or gay

Bisexual

Something Else (Go to A)

Don’t Know (Go to B)

Spanish: Usted piensa en sí mismo como…

[For men:] Gay

[For women:] Lesbiana o gay

[For men:] Heterosexual, o sea no gay

[For women:] Heterosexual, o sea no lesbiana o gay

Bisexual

Otra cosa (Go to A)

No sabe (Go to B)

A. English: [If ‘something else’ is selected] By something else, do you mean that…

You are not straight, but identify with another label such as queer, trisexual, omnisexual or pan-sexual

You are transgender, transsexual or gender variant

You have not figured out your sexuality or are in the process of figuring it out

You do not think of yourself as having sexuality

You do not use labels to identify yourself

You made a mistake and did not mean to pick this answer

You mean something else (Go to C)

Spanish: Cuando dice Otra Cosa, quiere decir que…

Usted es gay o lesbiana, pero se identifica más con otras clasificaciones como queer, multisexual, o trisexual

Usted es transgénero o transexual

Usted no sabe o está en el proceso de descubrir su sexualidad

Usted no piensa en sí mismo como teniendo una sexualidad

Rechaza personalmente todas las etiquetas para describir a su persona

Usted se equivoco y no quiso escoger esta respuesta

Usted quiere decir otra cosa [Go to 6c]

A. English: You did not enter an answer for the question. That is because you:

You don’t understand the words

You understand the words, but you have not figured out your sexuality or are in the process of figuring it out

You mean something else

Spanish: Cuando dice No Sabe, quiere decir que…

Usted no entiende las palabras

Usted entiende las palabras, pero no sabe o está en el proceso de descubrir su sexualidad

Quiere decir otra cosa

C. English: [If ‘you mean something else’ is selected]

What do you mean by something else? Please type in your answer

__________________________________

Spanish: ¿Que quiere decir por otra cosa?

Por favor escriba su respuesta:

________________________________

The sexual identity question tested by the QDRL demonstrated an overall marked improvement over questions that had been previously tested [24]. The goal was to develop a question that would not only reduce the rates of missing and ‘don’t know’ responses, but also help those who were answering to answer “more correctly”—that is, to reduce misclassified responses as well as reduce missing responses. To that end, three meaningful design principles were used: 1) use labels that respondents use to refer to themselves, 2) do not use labels that some respondents do not understand, particularly if those terms are not required by any other group of respondents, and 3) use follow-up questions to meaningfully categorize those respondents answering ‘something else’ or ‘don’t know’. These revisions were shown to be largely successful as the vast majority of respondents were able to select the category that best reflected their sexual identity. Of most import to the topic of this paper, the presence of the ‘something else’ category, and the subsequent follow-up options, was successful at helping transgender respondents more accurately identify themselves.

As shown in

Table 2. Responses to Sexual Identity Question.

Sexual Identity Category *	Frequency
Gay or Lesbian	5
Straight—that is, not gay	3
Bisexual	1
Something Else	12
Total	21

Note: * These are the English language translations.

, trans respondents fell across the spectrum of identifying as gay or lesbian, not gay or lesbian (and thus, straight or heterosexual depending on the language in which they took the survey), and something else (followed most typically by the response sub-option of ‘you are transgender, transsexual, or gender variant’). This complexity of response options from a single demographic is not viewed as problematic, however, as the question is intended to capture self-reported identity.

Many trans respondents referred to the gay community in broader, more encompassing terms than LGB or heterosexual respondents. Thus, a number of trans respondents conceived of the term “gay” as both an individual identity as well as an umbrella term for a larger community of sexual minorities (the exact composition of that community varied among respondents). One trans respondent said that although gay can specifically refer to a man who is masculine it can also be used to refer to “the whole community”. Another trans respondent wanted to choose the term transgender but since it was not available in the list of primary options, he chose gay because he felt that this was the closest option for him since it would include him in the LGBT community. Another trans respondent said that she thinks of the term ‘gay’ as being in the middle of a big circle of other terms like bisexual and transsexual and that gay is the word used to describe all of these things. She said that gay is the generic word used to describe all of these other terms but that it is not specific enough and she would not identify this way. Instead, she identifies specifically as transsexual.

Several interesting demographic themes emerged from the interviews as varying patterns of interpretation were found based not only along the lines of gender identification (discussed in depth below), but also along lines of education, age, and language of survey. There was a clear relationship between years of education and propensity to select ‘something else’ with those with a high school education or less being far more likely to identify as ‘something else’ than those with more than a high school education. It is also interesting to note that the only two respondents to identify as bisexual were both college-educated, identified their gender as male, and spoke English. Overall, younger respondents (under 40) were more likely to identify as ‘something else’ or ‘gay or lesbian’ while older respondents (over 40) were more likely to identify as ‘bisexual’ or ‘straight—that is not gay’. The improving climate for ‘something else’ identified people in pop and political culture in the United States today might help make sense of this trend.

4.3. Non-Trans Interpretation of Trans

One of the guiding principles behind the testing of this question was not to include words that would confuse other populations if they were not specifically needed by another population. This was not found to be a problem with the trans response options on either the gender or the sexual identity question. In neither case did a non-trans respondent inadvertently select one of those options.

Some respondents, especially those who did not identify as ‘something else’ had varying initial conceptions of what the ‘something else’ category could possibly mean or simply had no idea what it might imply. A heterosexual female, for example, said that something else made no sense to her because either you are straight or you are not. Another heterosexual respondent thought that “maybe they like dogs.” Another female respondent said that something else could be a hermaphrodite. She said that she knew a couple of hermaphrodites and that these are people born “with both sexes, both organs,” and then their parents decide if they want to raise them as a boy or a girl. Another respondent said it was for someone who does not know whether they like men or women and is the same as the ‘don’t know’ option. The most common understanding of the ‘something else’ category, however, was that it implied some variation of an understanding of transgender. One respondent, for example, said that something else is for those people who do not know what they want to be—male or female—and that they have not found their sexuality yet. Another respondent felt that maybe it was for people who did not want to openly identify as gay or who were transgender or “lost” and do not really know what they are. Others noted that it was a category for people who are not a lesbian or a homosexual. A gay male respondent said that “there are so many letters now” and so it gives people a chance to pick something different. Perhaps the most important finding of non-trans understandings of the something else category is that its presence did not increase response error. That is, these respondents did not choose this option because they understand that it was not for them. On the other hand, many trans respondents did choose this option, thus increasing response accuracy.

5. Intersection of Gender and Sexual Identity

One of the interesting, and perhaps most insightful, findings of this study was the relationship between how trans people identified on the gender question compared to how they identified on the sexual identity question. In general, results indicate that trans respondents made a stronger, but less clear, distinction between their gender and sexual identity than non-trans respondents. It should be re-emphasized that this data is not from a representative sample and, therefore, cannot be used to deduce larger population trends. It can, however, be used to help better understand patterns of interpretation among respondents.

Gender identity was shown to be a particularly prominent component of sexual identity for trans respondents. Several of the trans respondents noted that the first thing they looked for was a ‘transgender’ response option. This was certainly not the case for all trans respondents, as some chose ‘gay or lesbian’ or ‘straight—that is not gay’ without debate or hesitation. However, when failing to find this option, these respondents then chose ‘something else’ assuming that that is what it meant. This association might have been heightened by the fact that the gender question also asks if someone is male, female, or it is more complicated. Even several non-trans respondents felt that ‘something else’ was connected with the ‘it is more complicated’ category on the gender question. In both cases, the non-normative response was given a somewhat generic, catch-all heading. This might also help to explain why trans respondents saw a stronger association between their gender identity and their sexual identity.

While respondents did understand the differences between gender identity and sexual identity, more respondents identified as transgender in the sexual identity question than the gender identity question. Ultimately, this study highlights the complexity of these issues among individuals. The complexity is summarized well by a respondent who currently identifies as genderqueer but is considering a switch to identifying as transgender. She said, “If I were to transition into male there are some people who consider me straight but I don’t feel like I would fit into the cissexual identity [Note: ‘Cissexual’ is a term used to describe those who feel that their current gender identity matches that of the social expectations of their medically assigned sex at birth] of straight. So I guess I would go towards something else and have a very complicated sexual identity.”

As

Table 3. Gender Identity by Sexual Identity.

		Sexual Identity
Gender Identity		Gay or Lesbian	Straight—That is, Not Gay	Bisexual	Something Else	(Totals)
	Male	2	--	1	2	5
	Female	2	3	--	4	9
	It is more complicated	1	--	--	6	7
	(totals)	5	3	1	12	21

indicates, those who identified as ‘it is more complicated’ on the gender identity question were most likely to identify as ‘something else’ on the sexual identity question. Those who identified as male were least likely to identify as something else with those who identify as female falling somewhere in between. This suggests that a respondent who identifies outside of the gender binary is also more likely to identify outside of hegemonic sexual identity categories as well. It is also noteworthy that the only ‘bisexual’ response came from a respondent who identifies as male and the only ‘straight—that is, not gay’ responses came from those who identify as female. Those who identified as ‘it is more complicated’ on the gender question only selected either ‘something else’ or ‘gay or lesbian’.

One of the advantages of cognitive interviewing and follow up probing is that it allows researchers to gain deeper insight not only into the what of the response, but also into the why. Further probing revealed that at least four of the respondents would have chosen a trans option but because they did not see it (it was not in the original set of options but rather only as a sub-option under ‘something else’) and so they chose another option. Two of these respondents ended up identifying as ‘straight—that is, not gay’ and two of them as ‘lesbian or gay’.

One of the respondents who chose ‘straight—that is, not gay’ did so only after a long hesitation. He said that although he knows other people probably think of trans as more gay than straight, he does not identify as gay and so ended up not choosing it. Another respondent who also chose ‘straight—that is, not gay’ said she did so because she identifies as female and is attracted to men so that makes her straight. She said that if she had seen the trans option, however, she would have chosen that. She noted that she would never have gotten to that sub-option because she was very put off by the connotation of ‘something else’ and so she would likely not be identified as trans if that is how it is listed.

One of the respondents who chose ‘lesbian or gay’ said they would have picked trans right away but as it was not on the list, they did not feel that they had that option. Another respondent who chose ‘lesbian or gay’ said they use the term ‘transsexual’ to describe themselves “and nothing else”. She never uses the term gay to describe herself but as she did not see a trans option she felt that gay was the option with which she most closely identified.

Aside from the above misclassifications, there were also a number of other respondents who, although they did end up in the “right” category, said it would have been much easier for them if trans had been in the original list of options. One respondent noted that their “first instinct” was to choose trans. When they did not see this option, they ended up selecting ‘something else’ and then the trans sub-option.

Several of the respondents held a strong disassociation with the gay and lesbian community. Like many of the straight-identified respondents, their most salient sexual identity was not a direct association, but rather a “not-me” identity—that is, they defined themselves more by what they were not than by what they were [28]. Several trans respondents, for example, explicitly identified as “not gay” emphasizing that just because they are trans does not mean that they are gay. One respondent when asked to identify a trans identity stated that it is a transition from being a man to being a woman or vice versa but that this does not imply that you are gay or lesbian. It simply implies that you are trans. Another respondent said, “I cannot identify myself as either lesbian or gay because…because I am not a woman to say that I am a lesbian. And I also don’t want to say that I am gay because for me it’s a word that only pertains to homosexual behavior. So, I thought that I could find a word that would better pertain to how I more identify.” Another respondent explained, “I don’t consider myself to be gay because I feel like the term gay is intended for like gay men. And straight is I guess if you consider me to be female then the kind of guys I like I mean and they are guys are like straight guys that I’ve ever been with so…[…]…once next year is over [when she gets her surgery] I probably would say straight.” If the option for ‘something else’ had not been there, she said would have selected straight.

Building on the above, there are also many within the trans community who still more closely associate with the conventional dichotomy of gay and straight. A clear theme among many of these respondents was that whether or not they identified this way was directly related to where they were in their transitioning process. It is interesting to note that this transitioning process was defined as a physical one rather than a mental, emotional, or social one. One Spanish-speaking respondent, for example, said that they identified as gay because “I have not made changes to much of my body. So, I am gay”. Another Spanish-speaking respondent said that she does not identify as a lesbian because that is a term for women who like women, and as she does not like women, and so she cannot be a lesbian. She also does not identify with the word gay because that is a term for men who like other men and although she likes other men, she is no longer a man. She also said that she does not identify as transgender because she is not yet a transgender—who she defines as having made the full cross-over from one sex to another—but rather is in the process of transitioning genders. Indeed, for these respondents, unlike for many other trans respondents, their identity as trans was more about transitioning than about a stable identity. They see their current identities as transitional rather than fixed and permanent.

6. Conclusions

One of the advantages of cognitive interviewing is that it allows researchers to gain insight into the thought processes of respondents and thus they can take analysis beyond a cursory understanding of the statistical data. In this case, probing on the gender identity and sexual identity questions proved particularly useful to gain a better understanding of why certain respondents answered the way they did and to a noteworthy extent enabled a richer understanding of the data.

Overall, findings indicate that the response option for ‘something else’ was well understood by those who identified as such. The ‘something else’ option was the one most frequently chosen by trans respondents, who then most frequently selected the trans sub-option in the follow-up question. Overall, the data indicates that the presence of a trans category in the list of primary response options, however, would likely have a significant effect on how members of the trans community identify both their gender identity and especially their sexual identity on official surveys.

The findings from this study cannot help produce the perfect survey question to capture the trans population. Given the shifting, flexible, complex nature of trans identity, that is a goal that will be difficult to achieve. Until there is a nationally representative, random sample survey of, or including, the trans population, we will only have qualitative inferences. These findings can, however, move us several steps closer to understanding how to not only improve survey methodology related to the trans population and, therefore, to obtain “better” results, but also how to better interpret those results. An improved survey methodology and, more importantly, an improved means of making sense of that methodology are important first steps toward improving our understanding of the various issues facing the trans population today.