Mining Impersonification Bias in LLMs via Survey Filling

Abstract

In this paper, we introduce a survey-based methodology to audit LLM-generated personas by simulating 200 US residents and collecting responses to socio-demographic questions in a zero-shot setting. We investigate whether LLMs default to standardized profiles, how these profiles differ across models, and how conditioning on specific attributes affects the resulting portrayals. Our findings reveal that LLMs often produce homogenized personas that underrepresent demographic diversity and that conditioning on attributes such as gender, ethnicity, or disability may trigger stereotypical shifts. These results highlight implicit biases in LLMs and underscore the need for systematic approaches to evaluate and mitigate fairness risks in model outputs.

1. Introduction

The rapid adoption of Large Language Models (LLMs) in research, education, health care, human simulation, and decision-support systems is making it increasingly urgent to examine critical risks related to bias, fairness, representational harms, and interpretability of LLMs’ outputs [1,2]. Modern LLMs are indeed more than mere linguistic engines: they may implicitly encode and reproduce cultural priors, social norms, and biases present in their training data [3,4,5]. As LLMs are trained on large-scale web data, they inevitably encode the statistical regularities, social priors, and cultural assumptions embedded in those corpora. A growing body of research has documented that these models reproduce and sometimes amplify social biases along different dimensions, such as gender, race, religion, and disability [1]. Since existing classical methods for measuring stereotypes and bias in LLMs rely on manually constructed datasets that struggle to capture subtle patterns specific to demographic groups, recent work has turned to LLM-based simulation of personas to uncover stereotypes at the intersections of demographic categories [2,6]. In this context, a persona is a natural language portrayal of an imagined individual belonging to a given demographic group. By manually varying the demographic features of a target persona and comparing the resulting portrayals, it is possible to analyze how LLMs adapt their responses and thus reveal previously uncaptured stereotypical patterns. Prior studies, for example, show that LLMs frequently adopt stereotypical or exaggerated traits when conditioned on gender or ethnicity [2] or disability status [7] and that prompts involving marginalized identities can elicit outputs reinforcing harmful associations.

While previous studies have examined settings where demographic features are explicitly specified and the goal is to generate a portrayal of a persona based on those features, a different task remains largely underexplored: asking an LLM to impersonate a “generic” or “default” person without explicitly providing any demographic attributes. In this case, the model must not only generate the portrayal but also implicitly construct the underlying demographic features that define that persona. This raises the question about how LLMs conceptualize the “generic” or “default” person.

In this paper, we address this gap by simulating 200 United States (US) residents and asking them to answer a set of socio-demographic questions, with the goal of assessing whether LLMs tend to produce standardized profiles. While prior work has focused on fine-tuning LLMs to simulate survey response distributions for global populations [8], we emphasize that our approach uses LLMs in a zero-shot setting, as our primary objective is to analyze bias and stereotypes rather than to evaluate the alignment between generated profiles and real-world population data.

In more detail, in this paper, we address the following research questions (RQs):

RQ1 

Do LLMs tend to use some “standardized” profiles when asked to impersonate a “generic” person, systematically avoiding certain characteristics? Do these profiles substantially differ from one LLM to another?

RQ2 

Are there substantial changes in the impersonated profile by specifying some additional characteristics of the individual, like sex, ethnicity, or disability? Do these changes give evidence of possible stereotypes or biases in the models?

Our contributions are manifold:

C1 

We introduce a survey-based methodology for auditing LLM personas, enabling direct comparison between model-generated profiles and real-world population statistics.

C2 

We empirically demonstrate that LLMs often default to standardized impersonations that underrepresent demographic diversity.

C3 

We show that conditioning on demographic attributes can induce substantial and sometimes stereotypical shifts in impersonated profiles, highlighting the presence of implicit biases with potential fairness implications.

C4 

We publicly release a dataset of 6400 questionnaires filled by four popular LLMs when impersonating US residents.

This paper is organized as follows: Section 2 reviews related work, Section 3 describes the methodology used to address the RQs, and Section 4 presents and discusses the results. Finally, Section 5 summarizes the findings and concludes the paper.

2. Related Works

2.1. Theoretical Foundations in DEI and Social Science Frameworks

Understanding bias in language models benefits from grounding in theories from diversity, equity, and inclusion (DEI) and the social sciences. Intersectionality theory [9] shows that overlapping identities—such as ethnicity, sex, and class—interact to shape unique experiences of privilege and discrimination, calling for analyses that move beyond single-axis frameworks. Ref. [10] extends this with the “matrix of domination”, describing how inequality operates across institutions, culture, and interpersonal relations, all influencing how individuals and groups are represented in society, which is then reflected in data used to train AI models. Even seemingly neutral systems can thus reproduce structural racism [11], a fact also observed in recent NLP research [2] on “unmarked” or “standard” persona representation. From the perspective of cognitive social psychology, ref. [12] explains that implicit biases in individuals arise unconsciously through socialization in unequal and not balanced environments and contexts, which today can be the web and social media, then used as training material for language models. To counter this, participatory and community-centered approaches [13] emphasize involving marginalized voices in AI design. Together, these frameworks connect social theory with computational work on bias detection and mitigation. Our study builds on these perspectives and analyzes how a standard individual is represented by LLMs across different socio-demographic characteristics and whether the impersonated profile changes when additional traits—such as sex, ethnicity, or disability status—are specified.

2.2. Bias Detection and Mitigation in NLP

Language models are trained on massive collections of largely uncurated web-based data; thus, they are widely recognized as carrying the risk of reinforcing harmful societal biases when applied in real-world applications [1]. These harms include stereotypes [14], misrepresentation of cultures and societies [2], exclusionary or derogatory expressions against vulnerable people [15], misleading and toxic concept associations [16,17], and numerous other forms of bias [18]. The effects of these biases and stereotypes become unpredictable when applied to different downstream tasks, where it is demonstrated that historically vulnerable and marginalized groups are disproportionately harmed [19,20,21,22,23]. Growing recognition of these challenges has spurred extensive research aimed at assessing and mitigating bias, leading to the creation of evaluation metrics, benchmark datasets, and intervention strategies [1]. Despite their usefulness, these approaches have recently faced criticism due to the inherent trade-off between defining a fixed collection of stereotypes tied to particular demographic groups and achieving generalization across a wider spectrum of stereotypes and groups [24]. As a consequence, they fail to capture subtle and implicit patterns unique to specific demographic groups. Furthermore, they still often fail to account for the intersectionality between group attributes [25] or between broader structural oppressions [26]—aspects that recent social science research strongly recommends considering. To address these limitations, recent papers leverage LLMs’ capacity to simulate personas as a means of systematically modeling stereotypes associated with intersections of demographic categories [2]. This approach is not constrained by the boundaries of any pre-existing corpus and is capable of representing multiple dimensions of social identity. Accordingly, it offers a generalizable and adaptable framework for analyzing patterns in LLM generations with respect to diverse demographic groups. The next paragraph summarizes the state of the art in this direction.

2.3. LLM-Based Personas

Persona simulation involves prompting an LLM to generate a natural language portrayal of a hypothetical individual belonging to specific demographic groups [2]. This methodology has been employed both to assess the extent to which LLMs can independently reproduce different aspects of human behavior [27] and to model how humans might act within diverse social and political contexts [28,29,30]. In particular, ref. [31] measures how effectively an LLM can simulate personas when their demographic, social, and behavioral factors vary and indicates the challenging nature of the task, especially with the zero-shot approach. To address this, ref. [32] suggests enhancing the realism of personality trait representation by assembling a dataset of persona descriptions, which is then used to fine-tune LLMs to better align with human personality patterns. More recently, researchers have systematically compared zero-shot LLM-generated personas with real-world counterparts to uncover biases and distortions in LLMs’ representations [2,6,7]. Our work builds on this line of research by prompting the model to assume the role of a US resident and to complete a socio-demographic questionnaire. However, unlike prior studies in which socio-demographic attributes are predefined and the model generates a persona accordingly, our approach reverses the process: starting from a general persona category, we ask the model to produce its socio-demographic characteristics through the act of completing the questionnaire. We selected the questionnaire-filling task to address our research questions because it has garnered growing interest within the NLP community as a means of predicting human responses through virtual simulations. Moreover, by choosing this task, we also have the opportunity to compare the data generated by the models with the human-compiled data publicly available from resources such as the US Census Bureau, Pew Research Center, and YouGov US. Works in this direction are summarized in the next paragraph.

2.4. LLMs for Survey Filling

As LLMs demonstrate increasing ability to simulate human behavior at both individual and group levels, recent research is investigating their potential to generate accurate survey responses. The underlying motivation is that if LLM-generated responses reliably mirror those of real populations, they could accelerate social science research [8,33,34]. In [8,33], the authors specialize LLMs for the task of simulating survey response distributions. To enhance diversity in human survey responses, ref. [34] suggests examining how personality dynamics can influence responses, rather than focusing solely on socio-demographic factors. Our work differs substantially from theirs: while their goal is to train a system capable of accurately simulating the responses that real users would provide in a survey, here, we use the models without additional training, aiming instead to investigate whether LLMs tend to adopt “standardized” profiles when asked to impersonate a “generic” person, systematically avoiding certain characteristics.

3. Methods

To address the RQs presented in Section 1, we instruct the LLM to pretend to be an individual residing in the US and complete a survey about their socio-demographic features, thus merging the approach of [2], where LLMs are used to generate personas, and the idea of [8], where LLMs are used to fill socio-demographic questionnaires. To do so, we use the prompt reported in Appendix A. We did not perform extensive prompt optimization, as our goal was simply to show that bias exists and is easy to reveal using a straightforward prompt. The survey we provided is made up of multiple-choice questions drawn from established sources, i.e., US Census Bureau, Pew Research, and YouGov US. To avoid biases arising from first-token probabilities, as identified in [35], we instructed the model to write out the selected answer option from those available, rather than simply indicating the position of the choice. We explicitly specified “living in the US” in the prompt to enable alignment between the model’s responses and aggregated data from the cited sources. This approach, however, is not limited to the US and can be readily adapted to other countries. The questions presented are listed in

Table 1. List of questionnaire items answered by the LLMs.

ID	Source	Multiple-Choice Question
Q1	US Census Bureau	What is your age?
Q2	US Census Bureau	What is your sex?
Q3	US Census Bureau	What is your marital status?
Q4	US Census Bureau	What is your citizenship?
Q5	US Census Bureau	What is your race?
Q6	US Census Bureau	What is the highest degree or level of school you have completed?
Q7	US Census Bureau	What is your class of work/employment status?
Q8	US Census Bureau	What kind of work do you do?
Q9	US Census Bureau	How many hours do you work per week?
Q10	US Census Bureau	How much did you receive in wages, salary, commissions, bonuses, or tips from all jobs before taxes in the past 12 months?
Q11	US Census Bureau	Do you have a disability?
Q12	Pew Research	How do you describe yourself (gender)?
Q13	YouGov US	Overall, how satisfied are you with your life nowadays?

, while the answer options are provided in Appendix A.

For answering the first part of RQ1, we analyze the LLMs’ output to measure socio-demographic variability among generated personas, thus analyzing if LLMs use “standardized” profiles. For answering the second part of RQ1, we then analyze the most frequently returned profile for each model to verify if the most represented profile substantially differs among LLMs.

To address RQ2, we extend the base prompt by specifying additional characteristics of the impersonated profile. This is achieved by adding the following supplementary sentence directly after the initial sentence of the prompt:

“You are [PERSON]”.  

where “[PERSON]” can be “an asian person”, “an african american person”, “a white person”, “a person with a disability”, “a man”, or “a woman”.

To capture variability in LLM outputs, we administered the survey 200 times, with each run conducted independently, i.e., in separate chat sessions, to avoid potential bias from prior conversation turns or previously impersonated profiles, and we collected the distribution of the returned answers. To better interpret the observed distributions, we also report the actual statistical data on persons living in the US from the provided sources (surveys: US Census Bureau 5-Year estimates 2019–2023, Pew Research Center 2023, and YouGov US 2023): however, we remark that differently from studies like [8], our goal is not to understand how well the LLMs approximate the answer distributions of the persons living in the US but rather to understand if these models, by default, tend to impersonate some standardized profiles.

To understand if standardized profiles generalize across different LLMs, we experimented with two widely used proprietary systems (GPT and Gemini) and two open-source models that are relatively lightweight yet capable (Gemma and OLMo). Specifically, the following model versions were employed in the study, where the local models were executed in quantized form on an NVIDIA® GeForce RTX™ 4090 equipped with 24 GB of VRAM:

• OpenAI gpt-4o-mini (via Azure API)—henceforth “GPT-4o”;
• Google Gemini 2.5 flash (via Google API)—henceforth “Gemini-2.5F”;
• Google Gemma 3 27B (locally)—henceforth “Gemma-3”;
• AllenAI OLMo 2 32B (locally)—henceforth “OLMo-2”.

To preserve variability in the model outputs, we set the temperature to 1.0 for all runs while keeping all other parameters at their default values. Where appropriate, ${\chi }^2$ and Fisher’s exact tests were applied to assess whether differences in the responses across models and prompts or across models generated and real populations were statistically significant. We prioritized the use of the ${\chi }^2$ test when comparing model outputs with real-world distributions, provided that the minimum expected frequency requirement was satisfied. In cases where this assumption was not met—particularly when comparing models prompted with additional characteristics to the base version in RQ2—we employed Fisher’s exact test instead. The code and data—including the filled questionnaires generated by the LLMs, as well as the aggregated statistics and significance test results—are openly available at https://github.com/rospocher/impersonification-bias/ (accessed on 8 September 2025).

4. Results and Discussion

4.1. Regarding RQ1

Figure 1 presents the distributions of survey responses generated by the different LLMs, alongside the corresponding distributions from the original question sources. The CSV files with complete data used to produce Figure 1, are available in the GitHub repository for readers to consult. Although some variability exists, the figure shows that for many questions, model responses are relatively stable, exhibiting limited variation and strong consistency across systems. Overall, LLMs tend to portray demographically similar profiles associated with higher socioeconomic status—characterized by full-time professional employment, a bachelor’s degree or higher, and annual earnings above USD 50,000—while underrepresenting lower-income, part-time, unemployed, minority, or disabled groups.

In more detail, the reported age ranges are constrained—GPT-4o includes only 25–34-year-olds, while the others fall within 20–44. Gender distributions diverge somewhat: Gemini-2.5F and Gemma-3 skew heavily towards female, OLMo-2 skews towards male, and GPT-4o presents a more balanced mix. Marital status is likewise restricted, with responses limited to “never married” or “now married,” and GPT-4o reporting exclusively “now married”. Citizenship and ethnicity exhibit no diversity: all respondents are native-born and white. Educational attainment is notably high, especially for GPT-4o, Gemini-2.5F, and Gemma-3, which report only bachelor’s degrees or higher, whereas OLMo-2 includes a broader share of college-level education. Employment is exclusively in private companies, concentrated in “Professional, scientific, and management, and administrative and waste management services”, with several major industries being entirely unrepresented in the results. Work hours cluster around full-time (35+ h per week), and incomes are moderately high—Gemini-2.5F concentrated answers within USD 75–90 K, Gemma-3 and GPT-4o within USD 50–64 K, and OLMo-2 above USD 100 K. Disabilities are entirely unreported. Life satisfaction scores are above average for all models, with the highest being that for OLMo-2 (average: 8.06), followed by Gemini-2.5F (8.00), GPT-4o (7.97), and Gemma-3 (7.4), all quite above the survey data (5.96). Low life satisfaction scores are never generated, thus suggesting an optimism bias in synthetic responses, a fact already observed in [2,7].

Table 2. ${\chi }^2$ values obtained by comparing, for each question in the questionnaire, the distribution of answers produced by each model (GPT-4o, Gemini-2.5F, Gemma-3, and OLMo-2) with the aggregated statistics of the real-world population. Symbol “∗” denotes statistical significance (p-value $<0.05$). The distributions generated by the LLMs differ significantly from the real population across all the questions considered.

	GPT-4o	Gemini-2.5F	Gemma-3	OLMo-2
Age	1871.83 ∗	1262.05 ∗	1322.01 ∗	809.32 ∗
Sex	4.20 ∗	186.29 ∗	114.04 ∗	201.99 ∗
Gender	10.77 ∗	184.40 ∗	112.77 ∗	225.53 ∗
Marital Status	310.63 ∗	220.78 ∗	310.63 ∗	517.28 ∗
Citizenship	32.21 ∗	32.21 ∗	32.21 ∗	32.21 ∗
Ethnicity	115.46 ∗	115.46 ∗	115.46 ∗	115.46 ∗
Educational Att.	592.79 ∗	592.79 ∗	592.79 ∗	404.85 ∗
Employment Status	422.73 ∗	422.73 ∗	422.73 ∗	369.37 ∗
Industry	3163.54 ∗	1308.61 ∗	3163.54 ∗	2143.38 ∗
Hours Per Week	290.17 ∗	290.17 ∗	290.17 ∗	252.62 ∗
Earnings	1472.99 ∗	1694.22 ∗	2303.51 ∗	595.54 ∗
Disability Status	28.35 ∗	28.35 ∗	28.35 ∗	28.35 ∗
Life Satisfaction	977.18 ∗	1037.56 ∗	423.53 ∗	484.61 ∗

presents a comparison between the distributions of profiles generated by the LLMs and those observed in the real data, reporting the corresponding ${\chi }^2$ values and their statistical significance (p-value $<0.05$, denoted by “*”).

The results clearly show that the distributions of the LLM-generated populations differ significantly from the real ones across all the considered questions. This suggests that current LLMs struggle to accurately reproduce the demographic and attitudinal diversity present in real-world data, tending instead to flatten the population into a limited set of well-represented profiles.

When we examined the outputs for each repeated prompt in detail, we found that the models frequently generated the same complete profile (i.e., an identically filled questionnaire) multiple times, with Gemma-3 and OLMo-2 being the most and least repetitive, respectively. Indeed, for over 200 prompt repetitions, for GPT-4o, Gemini-2.5F, OLMo-2, and Gemma-3, we obtained only 10, 22, 75, and 7 differently filled questionnaires, respectively. The uniformity of the outputs across different LLMs is further confirmed by examining the most frequently returned profile for each model, shown in

Table 3. Most repeated profile produced by each model over 200 runs (c.f., “% Occurr. (out of 200)”). Common values among models for each question are highlighted with green cell background color and with the “(=)” symbol. In the table, “>= Bachelor’s” stands for “Bachelor’s degree or higher”; “private company” stands for “Employee of private company workers”; “Profess. […] services” stands for “Professional, scientific, and management, and administrative and waste management services”; “>=35 h/w” stands for “35 or more hours per week”. These values have been abbreviated to improve the presentation in the table.

	GPT-4o	Gemini-2.5F	Gemma-3	OLMo-2
Age	30 to 34 years (=)	30 to 34 years (=)	30 to 34 years (=)	30 to 34 years (=)
Sex	Female (=)	Female (=)	Female (=)	Male
Gender	Woman (=)	Woman (=)	Woman (=)	Man
Marital Status	Now married (=)	Now married (=)	Now married (=)	Never married
Citizenship	Native (=)	Native (=)	Native (=)	Native (=)
Ethnicity	White (=)	White (=)	White (=)	White (=)
Educational Att.	>= Bachelor’s (=)	>= Bachelor’s (=)	>= Bachelor’s (=)	>= Bachelor’s (=)
Employment Status	private company (=)	private company (=)	private company (=)	private company (=)
Industry	Profess. […] services (=)	Profess. […] services (=)	Profess. […] services (=)	Profess. […] services (=)
Hours Per Week	>=35 h/w (=)	>=35 h/w (=)	>=35 h/w (=)	>=35 h/w (=)
Earnings	USD 50,000 to USD 64,999 (=)	USD 75,000 to USD 99,999	USD 50,000 to USD 64,999 (=)	USD 100,000 or more
Disability Status	Without a disability (=)	Without a disability (=)	Without a disability (=)	Without a disability (=)
Life Satisfaction	8 (=)	8 (=)	7	8 (=)
% Occurr. (out of 200)	25.55% (51)	23.00% (46)	37.00% (74)	12.00% (24)

; the most frequently impersonated profile is largely the same across all LLMs, with only minimal variations.

Although the current experiments have certain limitations—such as the US-only context, the use of only 200 completed questionnaires per model, and the inclusion of just four LLMs—the low variability observed in the models’ responses when impersonating a US resident may indicate a well-established internal representation, as low variability often reflects high confidence or memorization [36].

4.2. Regarding RQ2

Figure 2, Figure 3, Figure 4 and Figure 5 show the answer distributions obtained by characterizing the impersonated profile according to gender, ethnicity, and disability, for GPT-4o, Gemini-2.5F, Gemma-3 and OLMo-2, respectively. For the scope of limiting the number of different conditions to compare, we considered only the three most popular ethnicities according to the US census data.

Statistically significant deviations (p $<0.05$, Fisher’s test) from the generic, non-specified generated profile are reported in

Table 4. Statistical significance (∗ means p-value $<0.05$) of the difference between the distribution of the answers generated by the models (GPT-4o,Gemini-2.5F, Gemma-3, and OLMo-2) when prompted to impersonate different profiles (an “Asian” person, an “African American” person, a “white” person, a “person with a disability”, a “man”, and a “woman”) and the distribution obtained with the generic prompt.

	GPT-4o						Gemini-2.5F						Gemma-3						OLMo-2
	Asian	African Am.	White	Disabled	Man	Woman	Asian	African Am.	White	Disabled	Man	Woman	Asian	African Am.	White	Disabled	Man	Woman	Asian	African Am.	White	Disabled	Man	Woman
Age	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗
Sex	∗	∗	∗	∗	∗	∗			∗		∗		∗	∗	∗	∗	∗	∗		∗		∗		∗
Gender	∗	∗	∗	∗	∗	∗			∗		∗		∗	∗	∗	∗	∗	∗		∗		∗		∗
Marital Status	∗			∗			∗		∗	∗	∗	∗		∗		∗						∗
Citizenship	∗						∗						∗						∗	∗		∗
Ethnicity	∗	∗					∗	∗					∗	∗					∗	∗		∗
Educational Att.				∗						∗						∗			∗	∗		∗		∗
Employment Status										∗						∗			∗	∗		∗	∗	∗
Industry	∗	∗		∗			∗	∗	∗	∗	∗	∗		∗		∗			∗	∗		∗		∗
Hours Per Week				∗						∗						∗			∗	∗		∗	∗	∗
Earnings	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗	∗		∗
Disability Status				∗						∗						∗				∗		∗
Life Satisfaction				∗		∗		∗		∗				∗		∗	∗	∗	∗	∗		∗		∗

and are denoted by an asterisk (*) throughout this section. Furthermore,

Table 5. Distributions of responses for “Age,” “Sex,” and “Ethnicity” in real data for individuals with disabilities (US Census) and in model-generated data obtained when prompting the models to impersonate a person with a disability. Chi-squared (${\chi }^2$) values comparing each model’s distribution with the US Census data are reported; ∗ indicates statistical significance at p < 0.05.

Age	US Census	GPT-4o	Gemini-2.5F	Gemma-3	OLMo-2
Under 5 years	0.003	0	0	0	0
5 to 17 years	0.078	0	0	0	0
18 to 34 years	0.133	0.58	0.04	0	0.275
35 to 64 years	0.365	0.42	0.96	1	0.645
65 to 74 years	0.185	0	0	0	0.08
75 years and over	0.236	0	0	0	0
Chi-squared (${\chi }^2$)		402.52 ∗	307.39 ∗	347.94 ∗	148.60 ∗
Sex	US Census	GPT-4o	Gemini-2.5F	Gemma-3	OLMo-2
Male	0.49	0	0	0	0.14
Female	0.51	1	1	1	0.86
Chi-squared (${\chi }^2$)		190.20 ∗	190.20 ∗	190.20 ∗	96.64 ∗
Ethnicity	US Census	GPT-4o	Gemini-2.5F	Gemma-3	OLMo-2
White	0.677	0.985	1	1	0.82
African American	0.135	0.015	0	0	0.035
Am. Ind. and Alaska Nat.	0.011	0	0	0	0
Asian	0.036	0	0	0	0.085
Nat. Haw. and Pac. Isl.	0.002	0	0	0	0
Some other race	0.051	0	0	0	0.04
Two or more races	0.090	0	0	0	0.02
Chi-squared (${\chi }^2$)		87.53 ∗	96.01 ∗	96.01 ∗	48.25 ∗

focuses on how LLMs and real-world distributions vary among different questions (i.e., age, sex, and ethnicity) for the population of people with disability. Exploring every combination of categories and questions would result in an overwhelming number of data, obscuring rather than clarifying key patterns. For this reason, we direct our attention to a particularly informative subgroup—individuals with disabilities—allowing for a more focused and insightful interpretation of the models’ behavior.

4.2.1. Overview

Overall, the plots in Figure 2, Figure 3, Figure 4 and Figure 5 indicate that prompting the models to impersonate profiles with specific socio-demographic characteristics influences the answers they generate. Beyond the expected effects on sex (and gender), ethnicity, and disability status—when the models are explicitly prompted to target these characteristics—other responses also appear to be influenced, particularly those related to age, earnings, industry, life satisfaction, and marital status. This pattern is confirmed by the data in Table 4, where the rows corresponding to these questions most frequently exhibit statistically significant differences compared with the generic impersonated profile.

Among the six specific impersonation the models were prompted with, the “person with a disability” one shows the greatest variability in responses and most frequently exhibits statistically significant differences compared with the “generic” impersonated profile (cf., Table 4), for all models, followed by those representing minority ethnicities (“African American” and “Asian”).

Across models, the results also indicate that GPT-4o, Gemini-2.5F, and Gemma-3 tend to cluster their answers around a few alternatives, whereas OLMo-2 produces more varied responses.

4.2.2. Age, Sex, and Gender

Across models, “Asian” profiles skew towards younger age (*), while “African American” and “white” often skew towards older age (*). “Disabled” respondents are consistently the oldest (*). Men tend to be modeled as slightly older than women (*). Gender balance varies greatly: Gemini-2.5F and Gemma-3 frequently generate female-dominated outputs, GPT-4o shows more mixed but still skewed cases (e.g., all disabled persons as mostly female), and OLMo-2 often skews towards male sex except for disabled persons.

4.2.3. Marital Status

Marital outcomes are stereotyped, often “now married” for default or majority groups (e.g., white, male, and generic), but “disabled” profiles tend toward divorce (*), and OLMo-2 uniquely has high “never married” rates across most groups. Gemini-2.5F and Gemma-3 preserve the “now married” dominance, whereas OLMo-2 diverges toward more varied marital statuses.

4.2.4. Ethnicity and Citizenship

Across GPT-4o, Gemini-2.5F, and Gemma-3, non-white groups appear only when explicitly prompted; otherwise, the models default to white identities—even for gender-specific prompts. For disabled profiles, non-white representation is minimal. OLMo-2 exhibits slightly more ethnic variety among disabled profiles but still predominantly generates white identities. Regarding citizenship, GPT-4o, Gemini-2.5F, and Gemma-3 tend to confirm “Native” except returning “Naturalized” when prompted for an “Asian” profile (*), with OLMo-2 frequently answering “Naturalized” or “Not citizen” for “Asian” (*), “African American” (*), and “with Disability” profiles.

4.2.5. Educational Attainment

Most models maintain high educational attainment for non-disabled groups (bachelor’s degree or higher). Disabled profiles consistently show lower educational attainment (*), most often at the “some college” or associate degree level. OLMo-2 produces the widest range, including lower educational levels also for “Women” (*), “Asian” individuals (*), and “African American” individuals (*).

4.2.6. Employment Status and Industry

Employment is overwhelmingly classified under private companies for non-disabled groups. Except for GPT-4o, disabled profiles diversify (*) toward non-profit, self-employment, unemployment, and/or under-16 categories. Industry concentration is heavy in “Professional, scientific, and management, and administrative and waste management services” for generic, male, white, and often “Asian” profiles. Educational services and health care dominate for “African American” (*) and disabled profiles (*), with OLMo-2 showing the greatest sector diversity (including “Other services” and “Wholesale trade”).

4.2.7. Working Hours and Earnings

Full-time hours (35+) dominate, but disabled profiles often work fewer hours or not at all (*). Income disparities appear in most models (*): men earn more than women (*), white individuals earn more than Asian and “African American” ones (*), and disabled profiles earn markedly less (*). Gemini-2.5F uniquely inverts this for “Asian” profiles (*), showing them with higher earnings than white and “African American” persons.

4.2.8. Disability

Only explicit prompts for disabled personas yield “with a disability” responses, with OLMo-2 more frequently associating disability with “African American” profiles (*). When prompted to impersonate a “person with a disability,” all models exhibit statistically significant differences from the generic profile across most questions. To contextualize these differences—and taking advantage of the aggregated data available from the US Census Bureau on “Age,” “Sex,” and “Ethnicity” for individuals with disabilities—we compared the models’ responses with the corresponding US Census distributions (cf. Table 5). In most cases, the models’ outputs align with the most frequent US Census category, thereby failing to capture the diversity of the real data. For instance, the models almost invariably associate disability with being female, whereas US Census data show that the proportions of males and females with disabilities are roughly equal.

4.2.9. Life Satisfaction

Baseline satisfaction is high in all models (often near or above 8), with disabled profiles being associated with consistently lower scores—Gemma-3’s disabled profile average (4.3) is especially stark (*). Differences by sex are minimal, though women sometimes score slightly higher than men (e.g., Gemma-3 (*)), and OLMo-2 (*) shows a small female disadvantage.

Summing up, the output for the different questions remains heavily skewed toward high socioeconomic status, white-collar, and white native-born identities, with diversity surfacing only when explicitly instructed—and even then, often constrained or stereotyped. The generated disabled profiles receive markedly different socioeconomic characteristics, often older age, lower education, reduced income, fewer hours, and lower life satisfaction. OLMo-2’s results are more heterogeneous and occasionally depart from the rigid patterns of other models but still reflect many of the same demographic biases.

4.3. Final Remarks for RQ1 and RQ2

LLMs tend to adopt standardized profiles when asked to impersonate a “generic” person and respond to demographic surveys. Even across multiple generations and with a relatively high creative temperature (1.0), a consistent and well-defined profile emerges, showing only minor variations (cf. Table 3):

A 30 to 34 years old; male/female; now married or never married; native US Citizen and white; having attained a bachelor’s degree or higher; working as employee of a private company in the professional, scientific, and management industry sector, for 35 or more hours per week, and earning at least 50,000 USD per year; without a disability and high life satisfaction (8 or 7, out of 10).

While some of these characteristics align in part with the most common responses in recent population surveys (e.g., the US Census), the experiment reveals clear exclusion biases: no young adults or elderly individuals, no personas identifying themselves as non-binary, no divorced, separated, or widowed statuses, no foreign-born respondents, no racial categories other than white, and no disabilities. When the generic persona is perturbed with specific attributes (e.g., sex, ethnicity, and disability), further notable patterns and exclusion biases emerge or are reinforced. For instance, disability is frequently associated with women—even in models where the generic profile is male. White profiles tend to be older on average than African American ones, who in turn are older than Asian profiles. Divorce appears only in responses for persons with a disability. Male and female personas are represented exclusively as white. Men earn substantially more than women, and white persons earn more than both Asian and African American ones. Finally, profiles representing persons with disabilities consistently report lower life satisfaction than others. These patterns may mirror both cultural stereotypes and certain associations occasionally observed in population studies. For example, the link between disability and divorce that appears in some generated profiles likely reflects narratives circulating in public discourse rather than empirical regularities. Although some research has reported higher rates of marital dissolution in households affected by disability—often attributed to contextual factors such as financial strain, limited support services, or health-related challenges [37]—these findings are neither universal nor deterministic. When such patterns are reproduced by models without context, they risk reinforcing existing stereotypes and framing complex social dynamics as fixed or inherent traits.

5. Conclusions and Future Work

Our study shows that LLMs tend to produce standardized profiles associated with higher socioeconomic status when asked to complete demographic surveys. Generic outputs are predominantly white, native-born, highly educated, full-time professionals with high life satisfaction and exclude minorities and historically underrepresented groups, such as individuals with a disability. Forcing the model to explicitly use some characteristics related to sex, ethnicity, or disability introduces some variation but often reinforces stereotypes, such as lower income or life satisfaction for profiles of individuals with a disability. These findings highlight that LLMs and AI systems in general reflect and may amplify existing societal biases, a fact already observed in social science studies [25,26,38], thus cautioning against their uncritical use in downstream tasks and realistic applications and advocating more equitable AI deployment. The reliance on biased LLMs can, in fact, lead to misinformed conclusions that may perpetuate stereotypes or overlook the needs of underrepresented communities [39]. For example, in education, AI tools used for student assessments or personalized learning may disadvantage students from marginalized backgrounds if the models are not trained on diverse datasets, potentially widening achievement gaps [40]. In health care, biased AI algorithms can result in misdiagnoses or unequal treatment recommendations, adversely affecting patient outcomes for minority groups [41]. In policymaking, decisions informed by biased AI systems can lead to the enactment of policies that unintentionally favor certain demographics over others, exacerbating social inequalities [42].

In future work, we plan to investigate more sophisticated prompting strategies and extend our experiments to a broader range of LLMs. We also aim to analyze more complex socio-demographic intersections to better understand how personas vary when certain attributes are fixed. In addition, we plan to extend this study to simulate individuals from countries beyond the US, comparing the generated survey responses with official demographic and survey data from national statistical agencies to address the current limitation of relying solely on US data. Finally, we are interested in developing methods to make LLM outputs more representative of minorities and diversity.

In conclusion, we remark that our findings are specific to the model versions tested. Since these models are frequently updated, future versions may display different behaviors.