Who Is to Blame for the Bias in Visualizations, ChatGPT or DALL-E?

Abstract

Due to range of factors in the development stage, generative artificial intelligence (AI) models cannot be completely free from bias. Some biases are introduced by the quality of training data, and developer influence during both design and training of the large language models (LLMs), while others are introduced in the text-to-image (T2I) visualization programs. The bias and initialization at the interface between LLMs and T2I applications has not been examined to date. This study analyzes 770 images of librarians and curators generated by DALL-E from ChatGPT-4o prompts to investigate the source of gender, ethnicity, and age biases in these visualizations. Comparing prompts generated by ChatGPT-4o with DALL-E’s visual interpretations, the research demonstrates that DALL-E primarily introduces biases when ChatGPT-4o provides non-specific prompts. This highlights the potential for generative AI to perpetuate and amplify harmful stereotypes related to gender, age, and ethnicity in professional roles.

1. Introduction

Ever since the public release in late 2022, generative artificial intelligence (AI) models, such as large language models (e.g., ChatGPT) and text-to-image models (e.g., DALL-E) have seen a rapid growth in popularity, leading to an increasing incorporation into the workflow of offices, schools and private life. Fundamental to the success was the ability to use plain-language prompts when communicating with generative AI models to receive plain language responses in return, as well as to use plain-language prompts to generate images. The key benefits of text-to-image models are that the images can be tweaked with structured prompts, often in an interactive fashion by refining the prompts, to develop complex images. A major concern, so far understated in the literature, is that fact that these text-to-image creations are copyright free and can be used by their creator in any medium and at any quantity desired. This makes these a cheap alternative to stock photography where reproduction fees are commonly charged based on the nature and quantity of their use. Consequently, it can be posited that the use of such generative AI-created imagery in the media and advertising will only increase. Underlying the rapid creation and dissemination of text-to-image generated visualizations is that, if created thoughtlessly, they will include visual biases derived from gender, age, ethnicity, occupational and other stereotypes which through their reproduction, will be widely disseminated and amplified.

Gender, age and ethnic (‘racial’) stereotypes that are pervasive social constructs based on generalisations and othering, which act as a shorthand in social communication [1,2] Such stereotypes when associated with professions and occupations are harmful, as they negatively affect people’s self-esteem and career prospects, and may result in various forms of prejudice and discrimination, both subtle and explicit) [3,4,5]. Therefore, people who do not conform to these stereotypical expectations often self-censor (‘stereotype threat’) and avoid pursuing such careers, thereby not only deny their own aspirations (with possible associated mental health impacts), but also further reinforce the existing stereotypes through underrepresentation. While image creations through art or photography involves a conscious decision which elements to include in the portrayal of a person and the image background, text-to-image generated depictions may include elements that were not included in the prompts but are embedded as derived from underlying biases or associations during the image generation process. As the biases and nuances may be subtle, the user generating the image may not be aware of the secondary messaging the image may convey. And even if the user is aware, the text-to-image generator may well be non-responsive to modification prompts [6].

Generative Ai models cannot be completely free from bias due a range of factors such as model architecture, the quality of training data, and developer influence during both design and training. Bias may arise from ideological perspectives, language selection during training, or reliance on sources that may contain outdated or racially biased information [7,8,9]. There is substantial evidence of gender stereotyping in generative AI responses [10,11], particularly in the reinforcement of normative identities and narratives, such as stereotypical binary gender roles [12,13,14,15]. Several studies have explored gender representation across various professions [10,11,16]. A cross-professional study found that 60% of the professions depicted aligned entirely with common gender stereotypes (e.g., mechanics being male and nurses being female) [17].

A substantial body of research has examined gender and ethnic bias in generative AI text-to-image applications such as DALL-E3, DALL-E4, Midjourney, and Stable Diffusion. Comparisons between Ai -generated text prompts and the resulting images suggest that biases are either introduced or amplified by the image generation algorithms [18]. In text-to-image Ai-generated images, covert bias through inadequate representation is widespread [19]. For instance, when gender is unspecified, Ai image generators predominantly depict men in “important” roles more frequently than women particularly in the medical field [20,21,22,23,24,25,26,27,28,29,30,31]. Similarly, unless explicitly directed otherwise, ChatGPT-generated prompts tend to produce images primarily featuring individuals of Caucasian appearance [20,21,22,23,24,25,26,27,28,29,30,32,33]. Other ethnic biases manifest in portrayals of African-American individuals in service roles [33], in contrast to Caucasian-looking characters, reinforcing patterns observed in textual analyses [34,35]. Beyond gender and ethnicity, Ai image generators also depict male and female figures with stereotypical facial and body features [24,30,36], often favoring younger individuals [21,24,28]. In single-shot prompting, representations of pregnant women or individuals with visible disabilities or impairments are notably absent [24].

This paper draws on an existing data set of 770 images that had been generated by ChatGPT-4o for an assessment of the representation of librarians [20] and curators [21]. The study investigates the biases present at the interface between large language models (LLMs) and text-to-image (T2I) applications. Specifically, it seeks to determine whether observed gender, ethnicity, and age biases in visualizations are introduced by ChatGPT-4o’s autogenerated prompts or by DALL-E’s visual interpretations of these prompts.

Despite the popular stereotypes of a librarian as a frumpy, middle-aged and unattractive spinster [20] and that of a nerdy bearded curator, conservative in dress and stuck in time rummaging through collections [21], the professions of librarians and curators are in fact gender neutral and thus are eminently suitable to examine underlying biases in generative AI. The scope of this paper is limited to the examination of these specific types of biases in the context of images generated by the ChatGPT-4o/DALL-E combination.

2. Methodology

This paper employs a mixed quantitative and qualitative analysis to examine 640 images of librarians and curators, as well as 200 women working in cultural industries generated from ChatGPT-4o prompts, analyzing the distribution of gender, ethnicity, and age in both the prompts and the rendered images as rendered by DALL-E.

2.1. The Data

Two data sets drawn on for this study had been generated for independent studies that examined the visual representation of librarians [20] and curators [21]. An additional data set contained representations of 200 women working in the cultural industries [37]. The studies for which these datasets were generated did not examine the causes of any observed biases.

The prompts given to ChatGPT-4o in the original studies were intentionally designed in an unconstrained fashion to avoid the injection of user bias and to prevent responses that are a priori biased towards the user’s perceptions. The following prompts were used in the two datasets:

“Think about [type of library] and the librarians working in these. Provide me with a visualization that shows a typical librarian against the background of the interior of the library.”

“Think about [type of museum] and the curators working in these. Provide me with a visualization that shows a typical curator against the background of the interior of the museum.”

ChatGPT-4o autonomously generated the detailed textual prompts for DALL-E to create the images, ensuring that no human intervention influenced the conceptualization or execution of the images. Each generated image was saved to disk, and the Ai-generated prompt was retrieved from the image panel (Figure 1) and stored alongside the image in a data file. After saving, the chat session was deleted to ensure a clean and unbiased generation process for new images, preventing any legacy information from influencing subsequent outputs.

2.2. Data Transparency

According to an established protocol [38], all images as well as the prompts that ChatGPT-4o/DALL-E used to generate the images discussed in this paper have been archived as formal data sets at the author’s institution and are readily accessible [37,39,40]. Given the size of the data sets, these are not reproduced in this paper and the reader is directed to the three sources.

2.3. Scoring

Scored were visual cues to gender (male/female), ethnicity and age. The scoring of age was based on facial representation alone, using on personal long-term assessment of imagery. Gender was classified as male/female. This binary classification excludes representations of non-binary individuals. This was adopted as none of the prompts specifically refer to non-binary.

Given that all images represented professionals at work, it was appropriate to divide the working age into three equal-sized bins (20–34, 35–49, 50–65), which were labelled ‘young’, ‘middle aged’ and ‘old’. The bin ‘middle aged’ (mid 30s to late 40s) functioned as the key classifier, defined by fine lines and wrinkles around the eyes and mouth (‘smile lines’/‘nasolabial folds’) and forehead, cheeks looking less full, and the midface appearing flatter (‘volume loss’) and skin looking less firm, around the jawline and neck, and under the eyes. Depictions with facial features lacking the middle-aged characteristics were classed as young, while depictions with white hair and/or pronounced wrinkles were classed as old. A second opinion was sought where it was not possible to unequivocally allocated into one of the three classifications. In addition, the presence of a beard for males, hair style (bob, open long hair, bun, ponytail) for women, as well as the presence of glasses/spectacles or a book were considered.

The coding of image prompts occurred based on specifications in the text (e.g., ‘middle-aged’, ‘woman’ etc.).

2.4. Statistics

Summary data and frequencies were established using MS Excel, while the statistical comparisons were established with MEDCALC’s comparison of proportions calculator (MedCalc Software, 2018, https://www.medcalc.org/).

2.5. Limitations

This study focuses on an examination of gender, age and ethnicity bias in the prompts used by ChatGPT and DALL-E and the resulting visualization by DALL-E. As both systems are continually upgraded, responses may vary over time. To account for this, the full responses have been documented in the datasets [37,39,40].

3. Results

Unless a user issues a very specific image generation prompt and restricts ChatGPT-4o to that prompt, ChatGPT-4o will interpret the user request and based on this, will autonomously generate the prompt that is issued to DALL-E for rendering. As the resulting image comes with a text of the prompt used (accessible via the image panel, see Figure 1), it is possible to examine the image generation sequence:

user prompt → ChatGPT-4o interpretation → DALL-E interpretation

The prompts autonomously generated by ChatGPT-4o can be specific, giving age or gender (Figure 2A); provide an inferred reference to age or gender (Figure 2B); or be silent on both age and gender (Figure 2C). DALL-E will then interpret and render these.

3.1. Gender

While the setting and paraphernalia describe a profession, even though some may be stereotypical (‘scientists wear lab coats’), the depiction of apparent gender is an important obvious signifier that influences how viewers respond to an image. Unless underlying conditions circumscribe the outcome, the representation of men and women should be equal in a larger dataset. In 77.8% of cases where the prompt autonomously generated by ChatGPT-4o specified a gender, that apparent gender was female.

Contextual identification to gender in the prompts was as obvious as the inclusion of ‘beard’ which would be uniquely male. In seven instances the choice of ‘tailored suit’ allowed for a positive identification, while in one stance the prompt text ‘tailored suit or a chic dress’ allowed DALL-E to generate either option (a male was rendered) (Table 1).

Table 1. Gender representation in prompts and rendered images.

		Apparent Gender as Rendered
		Female	Male	Total
gender specified	female	37	3	40
	male or female	24	28	52
	male	2	37	39
gender inferred via context	male (tailored suit)		7	7
	possibly male (blazer)	5	99	104
	dual gender (cardigan etc.)	15	11	26
no gender prescription		104	308	412
	Total	187	493	680

Table 1. Gender representation in prompts and rendered images.

		Apparent Gender as Rendered
		Female	Male	Total
gender specified	female	37	3	40
	male or female	24	28	52
	male	2	37	39
gender inferred via context	male (tailored suit)		7	7
	possibly male (blazer)	5	99	104
	dual gender (cardigan etc.)	15	11	26
no gender prescription		104	308	412
	Total	187	493	680

Figure 2. Examples of prompts autonomously generated by ChatGPT-4o and the resulting visualization by DALL-E. (A) prompt with specific age reference; (B) prompt with inferred gender reference (‘tailored suit’); (C) prompt that is silent on both age and gender.

Figure 2. Examples of prompts autonomously generated by ChatGPT-4o and the resulting visualization by DALL-E. (A) prompt with specific age reference; (B) prompt with inferred gender reference (‘tailored suit’); (C) prompt that is silent on both age and gender.

Almost a third of the prompts included the term ‘blazer’, 95.2% of were rendered as male (Table 1). The blazer is traditionally an exclusively male piece of attire, which predominantly remains to be so, although has also become part of the female professional wardrobe (‘power suit’) [41,42]. To clarify this, we can draw on a DALL-E generated image dataset of 200 women working in cultural and creative industries [37], which includes 80 librarians and curators. The term ‘blazer’ was included in 28.5% of the prompts (‘cardigan’ being 17.0%), although in 13% the inclusion of additional gender specific terms (‘blouse’, ‘skirt’) modulated ‘blazer’ and would have defined the output.

If ChatGPT attributes a blazer to both males and females, then the rendering of 95.2% of prompts as male suggests that DALL-E has an underlying bias towards directly or indirectly classifying cultural industry professionals such as librarians and curators as male. This is further underlined when considering the apparent gender representation in those instances where the prompt specification remained completely neutral. Here, 84.9% of visualizations rendered the professional as male (Table 1).

To assess whether the ambiguous prompt (‘tailored suit or a chic dress with subtle accessories’) would cause DALL-E to generate a balanced representation, ten iterations of the exact prompt text were run, resulting in eight images of a male and one image of female curator, with one image rendering a pair of curators of either gender. When the key sequence was reversed (‘chic dress with subtle accessories or a tailored suit’), DALL-E generated seven images of a male and one image of female curator, one image of pair of curators of either gender, and one image of a pair of male curators.

The question arises as to where the biases exhibited by ChatGPT and by DALL-E come from. To be relevant for either of the models, any comparative statistical data for the apparent gender profiles for librarians and curators (that form the basis of the analyzed data set) must predate the data cut-off for ChatGPT and DALL-E training data in September 2021. Considering English-speaking countries, in the USA, 82.5% of librarians and 60.2% of curators identified as women (2017) [43] while in Australia the figures were 83.8% and 67.6% respectively (2016) [44]. In the United Kingdom 76.1% librarians report as female [45], compared with 80% in Canada [46]. Taking the average gender balance of librarians and curators in these English-speaking countries as a guide, the picture emerges that approximately 70% of the workforce are women.

3.2. Age

Of the 680 autonomously ChatGPT4o-generated prompts only 22.6% specified an age, mainly ‘middle aged’ (

Table 2. Age representation in prompts and rendered images.

		Apparent Age as Rendered
		Young	Middle Age	Old	Total
age specified	young	19	10	4	33
	middle aged	16	65	15	96
	old	1	1	23	25
no age prescription	neutral	312	179	35	526
	total	348	255	77	680

). Among these, middle-aged staff dominate (62.3%), a proportion that is significantly higher than the representation of young staff (21.4%; χ² = 52.757, df = 1, p < 0.0001) and older staff (16.2%; χ² = 68.406, df = 1, p < 0.0001). Although ‘middle aged’ was specified, DALL-E rendered 16.7% of the ‘middle aged’ professionals as young and 15.6% as old. That a combined 32.3% are misrepresented is also very significant (χ² = 23.935, df = 1, p < 0.0001).

For the remaining 73.4% of the image prompts ChatGPT-4o was silent on age specifications and it was left to DALL-E to interpret and image. Contextually, as the position descriptions for both librarians and curators require a university degree, and it can be anticipated that the bulk of curators would belong to the middle-aged and old age groups. Among the prompts that provided no age specifications, however, the ages of professional as rendered, are dominated by young staff (59.3%), a proportion that is significantly higher than the representation of middle-aged staff (34.0%; χ² = 67.577, df = 1, p < 0.0001), which in turn are significantly more represented than older staff (6.7%; χ² = 120.814, df = 1, p < 0.0001). This demonstrates clearly that the prompt generation by ChatGPT overrepresents middle aged professionals (15.2%:71.7%:13.1%) while the visualizations by DALL-E overrepresent young staff (67.3%:26.6%:7.6%).

There are only few age profiles for librarians and curators that predate the training data cut-off in September 2021. When concatenated into the classes young (≤34), middle aged (35–54) and old (≥55), census data for the USA provide proportions of 22.0%:36.2%:41.9% for librarians (2017) [43] and 33.4%:28.1%:38.5% for curators (2020) [47]. When comparing the generative AI representations with USA data of staff working as cultural industry professionals (such as librarians and curators), then the overrepresentation of young staff (χ² = 31.285, df = 1, p < 0.0001) and underrepresentation of older staff by DALL-E is very significant (χ² = 29.070, df = 1, p < 0.0001), as is the overrepresentation of middle-aged staff (χ² = 31.174, df = 1, p < 0.0001) and the underrepresentation of older staff (χ² = 18.691, df = 1, p < 0.0001) by ChatGPT.

Returning to the additional 200 women working in cultural industries, in 92.0% of all instances ChatGPT-4o did not provide any age specification in its autogenerated prompts. Of these, DALL-E rendered 88.0% as young, 11.4% as middle-aged and 1% as old (

Table 3. Age representation in prompts and rendered images in a sample of 200 women working in cultural industries [37].

		Age as Rendered
		Young	Middle-Aged	Older	Total
age specified (ChatGPT terms)	young	1			1
	late 20s or early 30s	1			1
	mid-30s	4	2		6
	late 30s or early 40s		1		1
	mid-40s		2		2
	middle-aged	1	4		5
no age prescription	neutral	162	21	1	184
	total	169	30	1	200

).

When the rendered images of 200 women working in cultural industries were loaded into ChatGPT-4o to classify their appearance in 10-year age cohorts, the overwhelming majority (88%) was classified as belong to the 20–30-year age bracket (Table 3). Dissonances between the ChatGPT-4o classification and the visual classification carried out by the authors was 10.5%. Each of the dissonant images was independently assessed by a second party, confirming the author’s classification (

Table 4. ChatGPT-4o classification of the ages of 200 women working in cultural industries as shown in images generated DALL-E without age specification in the prompt.

		Age as Rendered
		Young	Middle-Aged	Older	Total
age cohort as identified by ChatGPT4o	10–20	2			2
	20–30	161	15		176
	30–40	6	13		19
	40–50		2		2
	50–60			1	1
	total	169	30	1	200

).

3.3. Ethnicity

In terms of ethnicity, only seven of the 680 images (1.03%) were depicted as other than Caucasian. These were six depicted with Asian features and one with Hispanic features [39]. All the 200 women in cultural professions were rendered as Caucasian. As none of the autonomously ChatGPT4o-generated prompts referred to ethnicity or diversity, this racial bias is solely due to the training data and algorithm used for DALL-E. That bias is extreme and exceeds that observed in other studies ^®.

Unlike the representation of ages and gender discussed above, the ethnic bias does reflect the state of the work force of librarians. Based on the statistics for the ethnicity of museum curators and librarians, 86.0% of librarians and 86.2% of curators in the USA were Caucasian (2017) [43]. Available figures for librarians in Canada (89%) [48] and the United Kingdom (91.9%) [45] are even higher. There are no data that allow us to correlate the observed bias in the ethnic representation of women in cultural professions with actual figures.

3.4. Glasses

A small number of generated images (11.3%) depicts librarians with glasses. Considering gender, DALL-E renders those prompts where ChatGPT-4o did not specify a gender very significantly more likely to be male than female (81.8%, χ² = 26.292, df = 1, p < 0.0001), Similarly, among the five prompts that were written as bi-gender, DALL-E rendered 80% as male (

Table 5. Presence of glasses by gender in prompts vs. rendered images.

		Gender as Rendered
		Female	Male	Total
gender specified	Male	—	100.0	16
	bigender	20.0	80.0	5
	female	91.3	8.7	23
no age prescription	neutral	18.2	81.8	33
	total	36.4	63.6	77

).

When considering the age distribution among people shown to be wearing glasses, there is a dissonance between the specified ages and the rendered ages, with rendered ages tending to be greater (

Table 6. Presence of glasses by age in prompts vs. rendered images.

		Age as Rendered
		Young	Middle Aged	Old	Total
age specified	young	40.0	30.0	30.0	10
	middle aged	9.1	45.5	45.5	11
	old	—	5.0	95.0	20
no age prescription	neutral	22.2	52.8	25.0	36
	total	16.9	36.4	46.8	77

). Among prompts where ChatGPT-4o did not specify an age, representations of people with glasses significantly more likely to be middle-aged than young (χ² = 7.091, df = 1, p = 0.0077) or old (χ² = 5.772, df = 1, p = 0.163). In contrast, the sample of 200 women working in cultural industries showed an overwhelming bias towards the depiction of young women where glasses were not mentioned in the prompts (89.7%) (

Table 7. Presence of glasses by age of persons as prompt specifications vs rendered images (in percent of age as rendered). Sample of 200 women working in cultural industries.

		Age as Rendered
		Young	Middle Aged	Old	Total
age specified	young	62.5	12.5	—	8
	middle aged	12.5	87.5	—	8
	old	—	—	—	0
no age prescription	neutral	89.7	9.0	1.3	78
	total	82.6	16.3	1.1	92

).

4. Discussion

The paper has shown that biases are introduced both at the initial generation of prompts by ChatGPT and the again when these prompts are interpreted by DALL-E. These biases result in imagery of individuals who are overwhelmingly Caucasian, predominantly male and young. It appears that these biases are inherent in the algorithms used by ChatGPT and DALL-E and that they come to the fore when these generative AI systems are required to fill in blank demographic or other details.

Based on the observations made in this study, we can propose the following flow of prompt interpretation and the nature and magnitude of the biases generated by ChatGPT and DALL-E (Figure 3). Following the user prompt, which in this study was specifically chosen to be unconstrained to avoid the injection of user bias and to prevent responses that are a priori biased towards the user’s perceptions, ChatGPT-4o interprets the prompt and generates a prompt for parsing to DALL-E. At this point biases can be created by direct specific mention of gender, age or ethnicity, or by inclusion of attributes in dress style that act as contextual directions to predicate a specific gender, age or ethnicity. Once parsed, that prompt is interpreted by DALL-E, which then applies its own ’interpretation’, thereby either correctly rendering the specific mentions (in most but not all cases), interpreting the contextual directions, or by applying its own interpretation to the unspecified prompt (essentially filling in the demographic blanks). It is at that point that DALL-E exhibits biases, such as an overrepresentation of males, of Caucasians and of young people.

As this paper has shown, ChatGPT-4o will generate image prompts to be parsed to DALL-E that nominate or infer a gender, Gender-specific prompts are not common compared to prompts where a gender, predominantly male, can be inferred contextually from specified clothing. As noted earlier, there is an abundance of evidence of gender stereotyping in generative AI responses, showing that the output is congruent with popular perceptions of professions-cum-gender stereotypes [10,11]. In this context, then, it is important to note that the stereotype of the librarian in major English speaking countries is that of an older, white woman [49,50,51] while the stereotype of a curator is overwhelmingly white and predominantly female [52]. Yet, as this analysis has shown, ChatGPT construes them contextually as male. While at this point in time the origin of the dissonance cannot be explained with a modicum of certainty, it is likely that underlying training biases in ChatGPT classify people in professional positions predominantly as male, irrespective of actual gender composition.

OpenAI used red teaming [53] to validate and moderate responses generated by ChatGPT with team members drawn from a wide range of expertise. While no specialists in museology or librarianship were represented, broadly relevant to the images analyzed here were red team members with self-reported domains of expertise in Anthropology, Sociology and Education [54]. The red team membership was biased towards educational and professional backgrounds (PhD’s or significant higher education/industry experience), coming from English-speaking, Western countries (U.S., Canada, U.K.) which would have influenced how they interpreted and flagged politics, values, and other representations of the model [55]. OpenAI does not comment on the gender and ethnic balance in the membership of its red team. While biases and some hallucinations in ChatGPT output are acknowledged, [45] no systemic biases in ChatGPT have been flagged by OpenAI [54,55,56,57].

The image creation process by DALL-E entails that a user-provided prompt is encoded through the auto-regressive transformer encoder, image tokens are sampled sequentially based on the decoder’s predicted distribution over the next token and the resulting sequences of image tokens are decoded through the VQVAE decoder [58]. The best of the multiple images is selected via CLIP (Contrastive Language-Image Pre-training) [59] and presented to the user [60]. DALL-E was trained on an image data set comprised of 250 million image and caption pairs [60] excluding graphic sexual and violent content as well as images of some hate symbols [61]. During the training phase of DALL-E, OpenAI used red teaming [53] to validate and moderate generated images. The red team membership was similar to that deployed for the evaluation of ChatGTP-4 [61,62].

OpenAI acknowledged in its system cards that biases will be “present in DALL·E 2 training data and the way in which the model is trained… how and for whom the system is designed; which risks are prioritized with associated mitigations; how prompts are filtered and blocked; and how uploads are filtered and blocked. Further, bias stems from the fact that the monitoring tech stack and individuals on the monitoring team have more context on, experience with, and agreement on some areas of harm than others” [61]. OpenAI further acknowledged that “default behavior of the DALL·E 2 Preview produces images that tend to overrepresent people who are White-passing and Western concepts generally…[and] tends to serve completions that suggest stereotypes, including race and gender stereotypes. For example, the prompt “lawyer” results disproportionately in images of people who are White-passing and male-passing in Western dress, while the prompt “nurse” tends to result in images of people who are female-passing” [61]. That bias persisted in DALL·E 3 as acknowledged by OpenAI: “by default, DALL·E 3 produces images that tend to disproportionately represent individuals who appear White, female, and youthful” [62] unless the prompts specifically moderate the output.

The data presented in this paper show that where the ChatGPT injected prompt specification remained gender neutral, DALL-E rendered the professional as male in 84.9% of visualizations (Table 1). This runs counter to the general observation that DALL-E renders people as female in unconstrained prompts [62], but confirms other studies that examined DALL-E visualizations [17,22,23,24,25,26,27,28,63]. On the other hand, the data in the foregoing study confirmed known DALL-E biases of youthfulness and a white/Caucasian bias [62,64], with young staff dominating (67.3%) the depiction of librarians and curators, while middle-aged and older staff were significantly underrepresented (Table 2).

One of the considerations could be the currency of the information that was included in the training data sets. For training data the development of ChatGPT drew on readily accessible and copyright free sources, which were comprised of old books and government publications, but also included internet text and Wikipedia pages, giving it some contextual ‘currency’ until the cut-off date of September [8,65]. While the same is more or less true for the development of DALL-E, that dataset may well be biased towards older imagery. A study by Shin et al., for example, indicated that generative diffusion models may have a preponderance of drawing on historical and cultural stereotypes when generating images based on prompts, rather than responding to direct and explicit prompt cues. All tested models, Stable Diffusion, DALL-E and Firefly, suffered from this [66].

It appears that while societal biases and stereotypes are ported into the algorithm via the training data, these are not caught and corrected during the red hatting phase of system evaluation. As noted elsewhere when assessing the cause of a persistence of bitcoin imagery in DALL-E generated images [6], the red hatting team may have normalized existing societal biases in their own daily lives and thus may not have been conscious of these when faced with imagery. Moreover, given that biases only become recognizable as a pattern when larger data sets are generated, it can be speculated that these biases could have escaped attentions during the red hatting phase as only a select number of images of a given configuration would have been generated. Given the harmful effects the perpetuation of social biases may have on individuals, these observations suggest that larger datasets of the same prompts need to be generated during the red hatting phase to identify and weed out the perpetuation of societal biases in generative AI content. It also suggests that the extensive representation of societal biases that is currently evident in generative AI content [10,11,12,13,14,15,16,19,24,25,27,28,29,30,31,32,33] is mitigatable with greater diligence during the internal evaluation period and preventable when completely new algorithms are designed and rolled out.

5. Conclusions

Considering a data set of 770 visualizations of librarians and curators derived from zero-shot prompts to ChatGPT this paper shows bias and initialization at the interface between LLMs and T2I applications which had not been examined to date. Comparing the prompts generated byChatGPT-4o with their visual interpretations by DALL-E, it explored biases of gender, ethnicity, and age. It could be demonstrated that most biases in visualizations were introduced by DALL-E in those cases where ChatGPT-4o generated non-specific image prompts, forcing DALL-E to fill in demographic blanks.

Users of the ChatGPT to DALL-E interface need to be aware of the biases introduced by both elements, by ChatGPT when interpreting the user request and creating the prompt to be parsed to DALL-E, and then by DALL-E when interpreting the ChatGPT-generated prompt. The more open ended the user-issued request, the more both ChatGPT and DALL-E are called upon to fill in the demographic blanks—with the result that their inherent, training-data derived biases come to the fore. Of potential concern for the novice, causal or uncritical user is the fact that both ChatGPT and DALL-E are subject to several inherent biases and that these perpetuate potentially quite harmful stereotypes. The generated visualizations create an impression that the professions are dominated by males, when in reality they are predominantly female, and that they are comprised of youthful individuals when in reality the median of age of staff is in the upper forties and lower fifties (depending on country). The only aspect of the DALL-E visualizations that somewhat reflects reality is the dominance of White Caucasians, even though the real proportions are considerably less than those depicted by DALL-E. While the stereotyping of the ethnic balance may reflect reality, they are deplorable as ethnic (‘racial’) stereotypes in relation to professions and occupations are damaging as they harm a person’s self-esteem and career prospects and may lead to discrimination (from subconscious to overt). In the case of curators and librarians, the ethnic pro-Caucasian bias reinforces the stereotype that such professions are not for non-whites, thus perpetuating the existing demographic bias and undermining ambitions members of other ethnic communities may have.

To overcome these algorithmic biases, users need to eschew the use of unconstrained single shot prompts. Rather seeking a ‘quick-fix’ solution by asking ChatGPT/DALL-E “provide me with a visualization that shows a typical XYZ”, they should instead issue a prompt that specifies gender, age and ethnicity. On the side of the developer, it would be desirable if the output DALL-E as parsed to ChatGPT could routinely present the user with two images, side by side, one depicting a female character and the other a male character. In terms of functionality this already exists as users are frequently asked which of two renderings, offered side-by-side, they might prefer.