Subtitling for the Deaf and Hard of Hearing, Audio Description and Audio Subtitling in Multilingual TV Shows

Abstract

Multilingualism in audiovisual productions has substantially increased in recent years as a reflection of today’s globalised world. While the number of publications looking at the phenomenon from the perspective of audiovisual translation (AVT)—especially interlingual subtitling and dubbing—has grown considerably in the last decade, there seems to be relatively little research on the rendering of multilingualism from the perspective of accessibility modes, namely subtitling for the deaf and hard of hearing (SDH) and audio description (AD). This article aims to investigate how multilingualism is rendered for deaf and hard-of-hearing as well as blind and partially sighted audiences, focusing on SDH and AD, as well as audio subtitling (AST). The study analyses a small corpus of TV shows available on Netflix and aims to highlight how multilingualism is made accessible both in SDH and AD. The products selected for the study had to satisfy three main criteria: they had to be a recent production, include the presence of an L1 (English) and one or more third languages and offer both intralingual SDH (closed captions) and AD. The results show that, even within the context of a single streaming platform, the strategies applied to deal with multilingualism seem to vary quite significantly both in SDH and AD/AST, ranging from neutralisation to L3 visibility.

1. Introduction: Multilingualism in Audiovisual Texts

This article aims to investigate how multilingualism in audiovisual productions is rendered for deaf and hard-of-hearing as well as blind and partially sighted audiences, focusing on subtitling for the deaf and hard of hearing (SDH) and audio description (AD), as well as audio subtitling (AST). While the number of publications looking at the phenomenon of multilingualism from the perspective of audiovisual translation (AVT)—especially interlingual subtitling and dubbing—has grown considerably in the last decade, there seems to be relatively little research on the rendering of multilingualism from the perspective of accessibility modes, namely SDH and AD.

Multilingualism in fiction—both in literary works and in audiovisual productions—has substantially increased in recent years as a reflection of today’s globalised world, as acknowledged by several AVT and film scholars (Bleichenbacher 2008; Díaz Cintas 2011; Dwyer 2005; Heiss 2004; Kozloff 2000; O’Sullivan 2011). The phenomenon of multilingualism has thus significantly affected cinematographic productions, which has led to an increased volume of audiovisual texts that display more than one language (Corrius et al. 2019). Audiovisual texts can present multilingualism to different degrees: a foreign language can be used occasionally, or different languages can constantly alternate. The use of multilingualism is often motivated by the filmmaker’s specific intentions, such as the attempt to offer a realistic representation of the world’s linguistic complexity (O’Sullivan 2011; Díaz Cintas 2011).

Since multilingualism on screen undoubtedly represents a challenge for translators, it is not surprising that in the last decade, there has been a significant rise in academic studies focused on the translation of multilingualism in audiovisual texts (Beseghi 2017; de Heredia and de Higes Andino 2019; Ranzato and Zanotti 2018; Rebane and Junkerjürgen 2019; among others). However, even earlier studies that justified the important role of multilingualism and language variation in fiction stimulated interest (see, for example, Corrius and Zabalbeascoa 2011; Bleichenbacher 2008; Delabastita and Grutman 2005; Sternberg 1981).

As Zabalbeascoa (2019, p. 19) put it, multilingualism in audiovisual texts—and generally in fiction—is part of the broader phenomenon of language variation, “characterised by the co-presence, mixing or code-switching of different languages, dialects, sociolects, creoles, made-up languages, diglossia, jargons, slang, and even special cases of speech disorders or temporary speech impediments”. L3 theory (Corrius and Zabalbeascoa 2011; Voellmer and Zabalbeascoa 2014; Zabalbeascoa and Voellmer 2014) is currently one of the most common approaches to account for the complex phenomena related to multilingualism in audiovisual texts and their translations. In this model, a distinction is made between L1 (the main source language), L2 (the main target language), and L3, which refers to any other language found in the source and target texts (Corrius and Zabalbeascoa 2011, p. 113). L3 is defined as “a deliberate use of expressive means (i.e., a language or language variety) that is distinguishable from most of the rest of the text, and this definition would include both foreign languages and dialects or other variations of a given language, including idiolects, sociolects and even special languages or varieties made up by the ST author” (Zabalbeascoa 2012, p. 324).

Up until now, multilingualism in AVT has been studied from different angles, focusing predominantly on interlingual translation (i.e., dubbing and subtitling) and on films (e.g., Baldo 2009; Beseghi 2017; Chiaro and De Bonis 2020; De Bonis 2014, 2015; Díaz Cintas 2011; Monti 2018; Parini 2015). However, on-screen multilingualism has become a widespread representational strategy in other audiovisual products as well, such as television series, which have recently become a very popular genre thanks to digital streaming platforms. The growing number of contemporary TV shows where different languages are spoken indicates that multilingualism is not related to a specific genre, channel, or platform (TV or web). On the contrary, it is displayed in a wide range of TV shows, such as fantasy, science fiction, mystery, crime, comedy drama, and comedy (Beseghi 2019).

Since a limited number of studies have focused on the challenges posed by multilingualism in SDH and AD, this article aims to explore how L3 is made accessible for both deaf and hard-of-hearing and blind and partially sighted audiences. It analyses a small sample of TV shows available on Netflix, looking specifically at which strategies are used in closed captions1 and AD/AST when L3 is involved in the original product. The results show that, even within the context of a single streaming platform, the strategies applied to address multilingualism seem to vary quite significantly both in SDH and AD/AST, leading to different effects, from neutralisation to L3 visibility. In this respect, it is important to consider the functions of L3 (Corrius et al. 2019) as well as its relevance in the TV show since they can determine the strategy to be used. Moreover, the way in which the TV show deals with multilingualism, for instance, by including part-subtitles for the exchanges in L3 with the translation in L1 or by introducing a diegetic interpreter, plays a fundamental role in the choice of strategy both in SDH and AD.

1.1. Multilingualism in SDH

As far as SDH is concerned, how can the presence of L3 be made visible to deaf and hard-of-hearing viewers? Szarkowska et al. (2013, 2014) proposed a set of strategies for rendering the presence of multilingualism in SDH using Sternberg’s (1981) model of linguistic representation, originally designed for literature but also applicable to audiovisual texts (O’Sullivan 2011). In Sternberg’s model, foreign languages in fiction can be represented at different levels, ranging from vehicular matching, which entails the presence of the foreign language without further information or explanation, to homogenising convention, which discards all language variation and prefers monolingual discourse. Szarkowska et al. (2013) applied this model to SDH and categorised five strategies for dealing with multilingualism:

• Vehicular matching;
• Translation + explicit attribution;
• Translation + colour-coding;
• Explicit attribution;
• Linguistic homogenisation.

The strategy of vehicular matching involves “showing the viewers the foreign language utterance by including the transcription of the original foreign text in subtitles, thus breaking the homogeneity of monolingual discourse usually created by subtitles.” This means that deaf and hard-of-hearing viewers can immerse themselves in multilingualism and experience L3 by seeing rather than hearing it. However, as pointed out by O’Sullivan (2011, p. 25), vehicular matching “has the potential to put considerable processing strain on the viewer”. The strategy of translation plus explicit attribution means translating L3 dialogue (either in L1 or L2) and telling the viewers that a foreign language is spoken, thus combining translation with explicit attribution. The strategy of translation plus colour-coding involves both translating and colour-coding L3 dialogue without naming the language. Both strategies somehow mark the presence of L3 by providing a label to explain what language is spoken or by colour-coding the translated L3 utterance. The strategy of explicit attribution involves “explicitly telling the viewers that a character speaks another language, as in [speaks French] and (IN FRENCH)” (Szarkowska et al. 2013, p. 294). Finally, the strategy of linguistic homogenisation, which lies at the opposite end of the linguistic representation continuum in Sternberg’s model, in SDH means “not signalling to deaf and hard of hearing viewers the presence of a foreign language in dialogue at all. Such subtitles would neither indicate (as in explicit attribution) nor show (as in vehicular matching) any trace of a foreign language to its viewers, hence reducing a more complex multilingual source text to a simple monolingual target text” (Szarkowska et al. 2013, p. 294).

The five strategies described by Szarkowska et al. can be applied both in the context of intralingual SDH, aimed at the primary audience of a given audiovisual product (i.e., viewers from the home country where the product is made) and interlingual SDH, aimed at a secondary audience when the audiovisual product is exported abroad. As noted by Szarkowska et al. (2013, p. 295), the decision whether an utterance in L3 should be transcribed, translated, explicitly attributed, or linguistically homogenised in SDH largely depends on the filmmaker’s original intention. If the filmmaker wants their audience to understand L3, pre-subtitles or part-subtitles2 are usually provided. This also means that SDH will include a translation for the utterance in L3. Conversely, if the filmmaker does not expect their audience to understand L3, thus leaving it un-subtitled, it is likely that it will also be left un-subtitled for the deaf and hard-of-hearing audience. This decision mainly depends on the role and function of L3 in the development of the plot and in the narrative in general.

As pointed out by Szarkowska et al. (2014), the role of pre-subtitling cannot be underestimated when selecting a strategy for SDH, both intralingual and interlingual; if the filmmakers do not include pre-subtitles for their primary hearing viewers, Szarkowska et al. (2014) suggest that the vehicular matching strategy should be preferred, as it allows the audience to become immersed in L3 through seeing what other viewers can hear. On the other hand, if pre-subtitles are provided, translation plus explicit attribution or translation plus colour-coding would be a more viable option. Furthermore, based on their reception study among deaf and hard-of-hearing Polish viewers regarding their preferences for SDH strategies, Szarkowska et al. (2014) show that even when viewers do not know the L3 involved, they favour more informative strategies where multilingualism is made explicit, such as vehicular matching and explicit attribution.

1.2. Multilingualism in AD and AST

Within media accessibility, multilingualism is, without a doubt, a challenge for AD: how can L3 be made accessible to blind and partially sighted audiences? As aptly noted by Corrius et al. (2019, p. 167), besides the common difficulties related to the translation of multilingual audiovisual texts, “specific problems must be faced when audio describing them, since audio description is not simply a matter of replacing visual images by verbal description”. Indeed, standard AD is not enough when dealing with multilingualism in audiovisual texts, which is why audio subtitling (AST), an aurally rendered version of written subtitles, can be used in combination with AD in order to provide information about L3 and its content.

Despite the many interesting aspects that are worth investigating in relation to multilingualism and AD/AST, up until recently there has been a limited number of studies on this topic (Benecke 2012; Braun and Orero 2010; Corrius et al. 2019; Iturregui-Gallardo 2020; Remael 2012; Szarkowska and Jankowska 2015a, 2015b). Iturregui-Gallardo’s (2020) research has focused on AST and how it can be used to reveal multilingualism to audiences with vision impairment or reading difficulties. Starting from Sternberg’s (1981) model of linguistic representation, Iturregui-Gallardo (2020) offers a categorisation of the strategies that may be used to reveal multilingualism through audio subtitles, similar to the classification proposed by Szarkowska et al. (2013) for SDH. In fact, although AST and SDH differ significantly in their formats, they can both provide varying amounts of information about multilingualism. The strategies for AST are categorised from more to less multilingualism-revealing:

• Vehicular matching;
• Selective reproduction;
• Selective reproduction + language information;
• Verbal transposition;
• Explicit attribution;
• Homogenising convention.

The strategy of vehicular matching involves leaving the original soundtrack without AST or any information about the L3. This means that the comprehension of L3 depends solely on the audience’s linguistic knowledge. Iturregui-Gallardo (2020, p. 493) observes that “such a strategy can be used depending on whether in the non-audio described version such utterance has been translated or not, or it can be conditioned by constraints such as time”. Selective reproduction means “the implementation of AST with a voice-over effect in the target language” (Iturregui-Gallardo 2020, p. 493). The L3 is heard in the background, which allows the audience to understand that it is a translation, but no further information on the language spoken is provided. Similarly, in selective reproduction plus language information, AST is provided with a voice-over effect, but the audio describer adds information on the language spoken. In verbal transposition, “AST goes beyond the translation by imitating and reproducing some of the patterns of the original foreign language spoken”, such as phonetic, semantic, and syntactic structures of the L3 (Iturregui-Gallardo 2020, p. 493). It can be used with a dubbing effect or a voice-over effect. Explicit attribution means providing audio subtitles with a dubbing effect after the L3 is announced by the audio describer. With the use of a dubbing effect, the result is closer to homogenisation; however, information about L3 is provided so that multilingualism is not completely erased. Finally, the strategy of homogenising convention is simply based on the full translation of the utterances in L3 by means of dubbing, thus deleting any traces of multilingualism and creating a monolingual discourse. No further information is provided by the audio describer, so the audience will be unaware of the presence of L3.

2. Materials and Methods

Taking into consideration the categorisations proposed by Szarkowska et al. (2013, 2014) and Iturregui-Gallardo (2020), this study analyses a small corpus of TV shows available on Netflix, focusing on how multilingualism is made accessible to deaf and hard-of-hearing and blind and partially sighted audiences. The products chosen for the analysis had to satisfy the following criteria:

• They are a recent production (from 2020 onwards);
• They include one or more third languages (L3) beside L1 (i.e., English);
• They provide both intralingual SDH (closed captions) and AD;
• They belong to different genres and display different third languages to different degrees.

The shows that were selected according to these criteria are Emily in Paris (Darren Star 2020–), From Scratch (Attica Locke and Tembi Locke 2022), Gentefied (Marvin Bryan Lemus and Linda Yvette Chávez 2020–21) and 1899 (Jantje Friese and Baran bo Odar 2022). The shows belong to different genres: romantic comedy, drama, comedy-drama, and mystery-science fiction, respectively. They display L3 to different degrees, in quantity and/or in number of different languages. Some of them contain L3 in certain scenes, and most dialogues are in L1, while in some others, L1 and L3 alternate continuously, creating a constant mixture of languages. Furthermore, the narrative, aesthetic, and ideological functions of L3 vary from show to show, mostly in relation to the plot and character portrayal as well as to the genre to which they belong.

The following paragraphs provide an overview of the presence and functions of L3 in the shows analysed. A useful categorisation of the possible functions of L3 in audiovisual texts is the one proposed within the Trafilm project and later applied to the MUFiTAVi project (Corrius et al. 2019; Corrius and Espasa 2022). The functions are (1) character portrayal; (2) stereotype; (3) plot (twist); (4) theme; (5) comedy/humour; (6) dramatic effect (i.e., dramatic impact of communication problems); (7) suspense; (8) metaphorical; (9) signalling otherness; (10) signalling the villain; (11) showing tolerance; and (12) metalinguistic function.

2.1. Emily in Paris

Emily in Paris is an American romantic comedy television series created by Darren Star for Netflix (Season 1, 2020; Season 2, 2021; Season 3, 2022). Set and filmed in Paris, the series follows Emily Cooper (Lily Collins), a young American woman from Chicago who moves to Paris to work for a well-respected French marketing firm while knowing little to no French. Cultures—and languages—clash as she struggles to succeed in her new professional and romantic life in the French capital. Emily in Paris is a show with a significant presence of L3: in all three seasons, English (L1) and French (L3) continuously alternate, while other third languages appear sporadically (e.g., Italian, Mandarin). Emily does not speak a word of French when she arrives in Paris. However, while the story develops, Emily slowly—and not without difficulty—starts to learn some French and uses both French phrases (code-switching) and words (code-mixing). Her use of French is also characterised by a strong American accent.

The most recurrent functions of L3 are comedy/humour and dramatic effect in the scenes in which L3 causes misunderstandings or funny situations; character portrayal, since Emily’s character is also represented through her linguistic journey; and finally, plot and theme. Indeed, the story is based on a character who moves to a different country where a different language is spoken, so learning the language becomes part of the plot. Other functions that seem to characterise the use of L3 in this show are stereotype, as language is used to reinforce certain stereotypical depictions of French people, as well as metalinguistic function in the bilingual wordplays.

2.2. From Scratch

From Scratch is an American drama mini-series created by Attica Locke and Tembi Locke for Netflix and released in 2022. The show tells the story of Amy Wheeler (Zoe Saldana), an American woman who falls in love with a Sicilian, Lino, who works as a chef in Florence. The show displays a moderate presence of L3, mainly Italian and Sicilian dialect, and occasionally Spanish. L3 is significantly more present in the episodes where the story takes place in Italy (Florence and Sicily), while it is less so when the two protagonists are in the United States. However, Italian is still used within the family by Lino, Amy, and their daughter, while Sicilian is spoken every time Lino phones his family back home and when they visit him in Los Angeles.

The most frequent functions of L3 are character portrayal; for example, the two main characters express their love through the Italian language; dramatic effect, in the scenes centred on the difficulty of communication due to language differences; plot and theme, since the story revolves around linguistic and cultural differences and the possibility of finding common ground through language; as well as stereotype, where language is used to underpin stereotypical depictions of Italian and Sicilian people.

2.3. Gentefied

Gentefied is an American comedy-drama television series (“a bilingual dramedy”3) created by Marvin Bryan Lemus and Linda Yvette Chávez for Netflix (Season 1, 2020; Season 2, 2021). Set in the Los Angeles neighbourhood of Boyle Heights, Gentefied follows the story of three Mexican-American cousins and their struggle to chase the American Dream while trying to keep their immigrant grandfather’s taco shop in business as the neighbourhood becomes more gentrified. The show is practically bilingual, displaying a constant alternation between English (L1) and Spanish (L3). Another third language that briefly appears is Mandarin. The grandfather, Casimiro, as well as other first- and second-generation immigrant characters, mainly speak Spanish, while the grandchildren predominantly speak English; however, they frequently switch to Spanish and use code-mixing.

The most frequent functions of L3 are character portrayal, when the language is used to represent different generations of immigrant characters; dramatic effect, in the scenes in which language differences create clashes or misunderstandings; as well as plot and theme, since the different immigrant experiences are depicted through language(s) and, in addition to the theme of gentrification itself, the show addresses the theme of code-switching in Latinx families.

2.4. 1899

1899 is a mystery/science fiction series created by Jantje Friese and Baran bo Odar for Netflix and released in 2022. Set in 1899, the series follows a group of European migrants travelling on a steamship from Southampton, UK, to New York City, U.S.A., in search of a new life. Despite being a German production, the primary target audience is English-speaking; in fact, the original version on Netflix is labelled “English”. 1899 is the show presenting the largest amount of L3 in the sample: besides English (L1), the third languages are French, German, Spanish, Portuguese, Danish, Polish, Cantonese, Japanese, and Norwegian. There is constant code-switching and code alternation in the show; in the first episode alone—which lasts 60 min—the characters change languages almost 60 times. The creators explained their decision to let the cast speak their own languages, rather than resorting to English, in an interview with The Hollywood Reporter:

There’s just an underrepresentation of the different European cultures, the different narratives from around the world. For a long time, even already in film school, we had this urge to change that, to really have characters from particular countries speak in their own voices, because language really defines character. If you want to have an authentic performance, it’s just better when actors perform in their own language.4

The most frequent functions of L3 in 1899 are character portrayal, since each character speaks the language they are supposed to speak based on their origin; in addition, L3 reinforces the plot and causes a dramatic effect. Indeed, the characters often communicate among themselves using their first language but without knowing the language(s) of the other(s), which contributes to enhancing the dramatic impact of the scene.

2.5. Guidelines for L3

As far as L3 is concerned, Netflix provides some guidelines for English SDH: when foreign dialogue is translated in the original version, subtitlers should indicate this by using “[in language]”, for example, “[in Spanish]”. Conversely, if L3 dialogue is not meant to be understood, subtitlers should use “[speaking language]”, for example, “[speaking Spanish]”. Subtitlers should always research the language being spoken; for instance, “[speaking foreign language]” should never be used. Accents or dialects should be treated in the same way, for example, adding “[in Spanish accent]”. Foreign words that are used in a mostly English line of dialogue do not need to be labelled but should be written in italics. Furthermore, subtitlers should always verify spelling, accents, and punctuation.5

Netflix also specifies its guidelines for AD/AST; when part-subtitles are used in the original product to translate L3 utterances, the same techniques used for on-screen text should be applied to introduce subtitles: explanation, name of the speaker, change in tone, multiple voices, and the description should read the subtitles verbatim. The original dialogue audio should be dipped to avoid confusion while still allowing the viewer to hear the original dialogue in the background. The audio describer should state “subtitles” when necessary to avoid confusion, for example, the first time they appear on-screen, and reintroduce them if significant time has passed before they appear again. For heavily subtitled content, multiple voices may be needed to help the audience differentiate the speakers. When song lyrics are plot pertinent and have been pre-subtitled, they should be read by the AD voice. They should not be sung but be timed to fit within the rhythm of the music as much as possible while allowing key phrases of the original to be heard. If the original lyrics are not subtitled but are plot pertinent, the audio describer should treat them as dialogue and avoid speaking over them.6

3. Results

This section presents the results of the analysis of the strategies used to deal with L3 in the closed captions (SDH) and AD/AST provided on Netflix for the TV shows selected. The analysis applies the categorisations proposed by Szarkowska et al. (2013, 2014) and Iturregui-Gallardo (2020), respectively.

3.1. Emily in Paris

Emily in Paris consists of 3 seasons7, for a total of 30 episodes, which last from 25 to 39 min. The third season, for a total of ten episodes, has been analysed in this study. This choice is justified by the fact that in this season Emily has become more integrated in the Parisian setting and her knowledge of French has slightly improved, which also implies a greater presence of L3. In this season, there is a main L3, French, while another language, Italian, is spoken only on one occasion.

In regard to SDH, the principal strategy observed in the ten episodes analysed is translation plus explicit attribution, whereby L3 is translated into L1, telling the viewer which language is spoken (see Example 1).

Example 1.

 

[in French] Have a good day. (Season 3, Episode 1)

The strategy of translation plus explicit attribution is also used for songs in L3; a translation of the lyrics into L1 appears on the screen, introduced by the subtitle “[singing in French]”. However, this strategy is applied exclusively to songs that are sung by a character (i.e., Mindy), while songs in L3 that are part of the soundtrack and are not pre-subtitled are neither translated nor transcribed in the subtitles but simply introduced by a label reporting the title of the song and the name of the singer.

In general, the strategy of translation plus explicit attribution reveals the presence of L3 without actually showing it. However, in some cases, L3 words or expressions are inserted in the translated subtitle through vehicular matching, thus creating a mixture of L1 and L3 (code-mixing), as can be seen in Example 2.

Example 2.

 

How’s everything going over there,/mon chéri? (Season 3, Episode 3)

Vehicular matching, which actually shows L3, is used both for single words or expressions (code-mixing) and entire phrases. For example, the French word bonjour appears 35 times in the subtitles, merci 27 times and bonsoir 17 times. Other French words or expressions which are found in the subtitles include greetings such as au revoir, bonne journée, bonne soirée, salut; appellatives such as madame, mademoiselle, monsieur, maman, mamie, ma chérie, mon chéri, mon amour; and common expressions such as bien sûr, bonne chance, bonne idée, ça va, excusez-moi, mon Dieu, pardonnez-moi, très bien, voilà. Such words and phrases sometimes appear in a subtitle which is written entirely in L3, as shown in Example 3:

Example 3.

 

- Bonne soirée.

- Merci.

(Season 3, Episode 9)

Code-mixing is, in fact, a distinctive feature of Emily in Paris, where both French and American/English characters mix the two languages, even in creative ways. The tendency in the closed captions is thus to maintain this characteristic (see Example 4).

Example 4.

 

Pardonnez-moi,/but my French has gotten très better. (Season 3, Episode 4)

Vehicular matching is also found when diegetic interpreting occurs in the narrative. For example, in Episode 4, Luc uses a French expression which is transcribed in the subtitle, and Julien explains it in English for Emily (Example 5). In Episode 5, Mindy uses a French expression while talking to Emily and then translates it into English for her friend (Example 6).

Example 5.

 

We have an expression in French./It’s “reculer pour mieux sauter”.

Step back to jump better.

Example 6.

 

Okay, Emily Jane Cooper,/occupe-toi de tes oignons.

Occupy your onions?

Watch your onions./It means stay out of it.

Another strategy that can be found, though to a lesser degree, is linguistic homogenisation. While the presence of L3 is generally addressed through translation plus explicit attribution, in some instances, utterances in L3 are translated into L1 without signalling the change of code. The inevitable result is that deaf and hard-of-hearing viewers will not be aware of the presence of L3 in these instances. Finally, the strategy of explicit attribution is found only once, when two characters briefly speak Italian and no translation is provided, only the subtitle “[both speaking Italian]” (Episode 9). Although Netflix’s guidelines recommend specifying the presence of accents, no indication is given about this aspect (e.g., English with a French accent or French with an English accent) in the subtitles.

By comparing the original version with the one with closed captions in English, it can be noticed that whenever part-subtitles are provided, translation plus explicit attribution is applied. However, when part-subtitles are absent in the original version, different strategies are found in the closed captions: translation plus explicit attribution, vehicular matching, explicit attribution, and linguistic homogenisation.

As far as AD is concerned, L3 is made present through two main strategies: vehicular matching and selective reproduction. The first strategy is applied when no part-subtitles are provided for L3 in the original version. In this case, the audio-described version keeps the original soundtrack intact without adding any AST or information about L3. The second strategy, which consists in the addition of AST with a voice-over effect, is used consistently when part-subtitles appear in the original version. Since the L3 can be heard in the background and can also be clearly heard when AST is not provided, no information on the language spoken is added, leaving the audience to infer what it is. AST is introduced by the word “subtitles”, which is repeated whenever a new set of subtitles is read, and by the name of the character speaking (i.e., name insertion strategy, Szarkowska and Jankowska 2015a). The text of the AST is generally verbatim (i.e., identical to part-subtitles), although in a few cases, it differs slightly to make it more suitable for oral delivery. The audio subtitles are read by two voice talents, which are gender-matched, while a different voice reads the AD.

AD also needs to account for on-screen text, for example, when characters exchange text messages. When messages are written in French, the original version provides a translation in L1, which appears not in the form of simultaneous part-subtitles but consecutive ones in the same format as the original message (i.e., on the phone screen). The AD voice, in this case, only reports the message translated into L1. This means that the blind and partially sighted audience will not have access to the bilingual text which appears on the screen, only to the translated one (Example 7).

Example 7.

 

In the living room, she downs a glass of wine, then sends a text message: she writes ‘any chance you could come to Paris for a few days?’ (Season 3, Episode 4)

As for L3 songs which are pre-subtitled, selective reproduction is used. The translated lyrics appearing on screen are read by the AST voice, introduced by the word “Subtitles” and the name of the character singing (Mindy).

3.2. From Scratch

From Scratch consists of a single season of eight episodes lasting from 49 to 58 min. In the show, two main third languages are spoken, Italian and Sicilian dialect, and some Spanish is occasionally present.

Concerning SDH, the strategies used to deal with L3 throughout the series are explicit attribution, vehicular matching, and translation plus explicit attribution. In the first episode, explicit attribution is the most common strategy. Amy arrives in Florence knowing little Italian, and she meets some Italian people, among whom Lino. During their first encounter, they exchange some phrases in Italian before switching to English. Here, as in many other instances throughout the episode, the captions do not translate the content of L3 utterances but simply indicate the language being spoken; the label “[speaking Italian]” is used to refer to multiple L3 utterances while “[speaks Italian]” is used to indicate a single L3 utterance. Although it is true that part-subtitles are not provided in the original version either, the difference is that hearing viewers can hear the dialogue in Italian and perhaps understand it even though they are not expected to. On the other hand, deaf and hard-of-hearing viewers are simply told that the characters are speaking Italian. However, in some cases, diegetic interpreting is an aid to comprehension. For example, when Amy first meets Lino, he asks her in Italian how she is enjoying Florence (“Come ti trovi a Firenze?”). This utterance is not translated in the subtitle, but immediately afterwards, Amy tries to translate the sentence, helped by her Italian friend (Example 8).

Example 8.

 

Uh… [speaks Italian]

Uh… how do I find myself?

- How are you enjoying Florence?

- Right. Certo.

Moreover, while explicit attribution appears to be preponderant especially in the first part of the episode—when Amy is more insecure, does not feel at ease speaking Italian and does not understand everything—in the second part, when Amy interacts more in Italian with the other characters, the strategy of vehicular matching increases. By transcribing L3 in the subtitles, the latter strategy contributes to making L3 more visible and tangible to deaf and hard-of-hearing viewers. Vehicular matching is found in closed captions both for single words in L3, which are often mixed with L1 (Example 9) and for entire phrases or utterances in L3 (Example 10).

Example 9.

 

No, sul serio, tell me what I am to you. (Episode 1)

Example 10.

 

Per piacere, un cappuccino. (Episode 1)

Another significant element of L3 in From Scratch is found in music, more specifically in Italian songs, which are part of the soundtrack. In the first episode alone, seven Italian songs by different Italian musicians are played, and the lyrics perfectly reflect the atmosphere of the scene and the characters’ emotions. However, since the lyrics are not subtitled in the original version, they are neither translated nor transcribed but introduced by a label in the closed captions (Example 11).

Example 11.

 

[“Per ricominciare” by Mina playing] (Episode 1)

From Episode 2 onwards, the most common strategy becomes translation plus explicit attribution, followed by vehicular matching and explicit attribution. The majority of L3 utterances are translated into L1, and the language is pointed out in the subtitle, as can be seen in Example 12.

Example 12.

 

[in Sicilian] Mama, I made/the risotto with almonds and broccoli.

It’s not like yours.

[mother] And how is the work? (Episode 2)

Vehicular matching is used in particular for short phrases (e.g., mangia, ti amo) and expressions such as greetings, responsive expressions, and appellatives (e.g., ciao, arrivederci, piacere, certo, grazie, prego, scusa, pronto, sì, amore, mamma, babbo, nonna), as well as for the description of dishes and recipes (Example 13). Moreover, as in Emily in Paris, vehicular matching is sometimes used in combination with translation plus explicit attribution. This means that the translated subtitle retains some L3 elements (Example 14).

Example 13.

 

[Lino] Insalata di arance e olive,/purea di fave e pane fritto,

panelle,

bacelli di fave alla brace,

arancine con riso verde e piselli.

And spaghetti col pesto alla trapanese. (Episode 4)

Example 14.

 

[in Italian] I’m tired, amore. (Episode 2)

Vehicular matching is used more frequently for Italian words, phrases, and utterances, while Sicilian is usually rendered using translation plus explicit attribution or simply explicit attribution: “[speaking Sicilian]”. However, in a few cases Sicilian is transcribed in the subtitles (Example 15).

Example 15.

 

Chistu co’u cappieddu è u’ patri di Amy. (Episode 8)

Although vehicular matching is not the most frequent strategy in the eight episodes, it contributes to highlighting the presence and enhancing the visibility of L3. It is mainly found when part-subtitles are not provided in the original version. There are no cases of linguistic homogenisation. Regarding the presence of accents, no indication is given (e.g., English with an Italian accent or Italian with an American accent) in the subtitles.

As for AD, the main strategy in the first episode is vehicular matching, which leaves the original soundtrack without AST or any information about the L3. This choice probably depends on two factors: firstly, in the non-audio-described version, such utterances have not been translated for the primary audience; secondly, in some cases, diegetic interpreting is used. In the following episodes, however, selective reproduction (AST with a voice-over effect) becomes the principal strategy to deal with L3. It is interesting to note that the audio subtitles are not introduced either by the word “subtitle” or by the name of the character. This means that the audience will have to infer this information in addition to the spoken language. Useful elements in this respect are the descriptions provided by the AD voice (e.g., “Filomena answers”), which can help understand which character is about to speak, as well as the use of the voice-over effect, which can help identify which language is spoken by which character. However, the distinction between Italian and Sicilian may not always be so clear. As in Emily in Paris, the audio subtitles are read by two voice talents, one per gender, while a different voice reads the AD. Concerning L3 songs, since they are not pre-subtitled, they are not translated in AD either.

3.3. Gentefied

Gentefied consists of two seasons, for a total of 18 episodes which last from 25 to 34 min. In this study, the first season has been analysed for a total of ten episodes. As mentioned before, the show displays a significant presence of L3, mainly Spanish, which constantly alternates with English (L1). In Gentefied, Spanish is mostly spoken by Mexican immigrants of first and second generations, while the third generation of Mexican-Americans tends to prefer English but often mix it with Spanish (i.e., Spanglish).

In terms of SDH, the most frequent strategy is translation plus explicit attribution. However, somewhat surprisingly, linguistic homogenisation is frequent, specifically in dialogues characterised by code-switching, where the continuous switch between L1 and L3 is not always signalled in the subtitles. In such cases, the audience is led to think that the subtitle is not a translation from L3 but an account of the L1 dialogue, thus erasing multilingualism. To illustrate this point, let us consider a scene in Episode 1 when one of the three cousins, Ana, has a fight with her mother, Beatriz. The subtitles do not indicate the various switches between English and Spanish (Example 16):

Example 16.

 

Where are my paints, Amá? (Ana, in English)

Oh, well, who knows? (Beatriz, in Spanish)

Maybe they went on vacations. (Beatriz, in English)

My, I wonder what it must feel like/to take a vacation. (Beatriz, in Spanish)

Tell them to send me a postcard. (Beatriz, in Spanish)

Because if you hadn’t noticed… (Beatriz, in Spanish)

I am stuck here killing myself/while you’re out there playing artist! (Beatriz, in Spanish)

Why can’t you just support me? (Ana, in English)

Your grandfather can’t pay/your rent anymore. (Beatriz, in Spanish)

Say goodbye/to your personal Don Francisco! (Beatriz, in Spanish)

It’s time you get a real job, mijita. (Beatriz, in Spanish)

I have a job. (Ana, in English)

A full-time job, Ana! (Beatriz, in English)

Another strategy that is seen, though to a lesser degree, is vehicular matching, which is used, for example, for entire utterances in L3 (Example 17).

Example 17.

 

Casimiro, qué linda familia. (Season 1, Episode 1)

It should be noted that when the closed captions display code-mixing, it is either because it characterises the original dialogue (Example 18) or because vehicular matching is used in combination with translation plus explicit attribution (Example 19):

Example 18.

 

Hombre,

I mean they love el tío Erik. (Season 1, Episode 1)

Example 19.

 

[in Spanish] What a great idea, mijo. (Season 1, Episode 1)

The least common strategy is explicit attribution, which is found especially when the dialogue is not clearly audible, for example, “[muttering in Spanish]”, “[overlapping Spanish chatter]”, and when L3 utterances are not pre-subtitled, for instance “[speaking Mandarin]”.

The soundtrack of the show blends songs in English and in Spanish. However, while the lyrics of the songs in L1 are transcribed in closed captions, the songs in Spanish are introduced by a label, such as “[“La Mentira” by Armando Garzón playing]”, with no transcription or translation. The exception to this is a song in Episode 10 (“Lo Que Siento” by Cuco), whose lyrics, which are half in English and half in Spanish, are wholly transcribed in the subtitles. Moreover, there are a few songs that are sung by characters; however, only in one case are they translated in the closed captions, although the lyrics are not subtitled in the original version, quite inexplicably, since they are plot-pertinent. Regarding the presence of accents, no indication is given (e.g., English with a Spanish accent or Spanish with an American accent) in the closed captions.

As far as AD is concerned, the main strategy is selective reproduction. AST is provided for the majority of L3 utterances, based on the presence of part-subtitles, which are read almost verbatim, with sporadic changes. The audio subtitles are occasionally introduced by the word “subtitle”, but the name insertion strategy is not used. Furthermore, the same voice reads both the AST and the AD script, modulating her voice in order to help the audience differentiate between the two modes. Surprisingly, although the AST normally reflects the presence of part-subtitles, there are some cases where the audio describer reads a translation for L3 words or phrases which were not pre-subtitled. For example, in Episode 7, the audio describer reads “women” when a character (Yessika) uses code-mixing: “Two weird brown mujeres?”. However, it is difficult to identify the criteria for such choices, as on other occasions, audio subtitles are provided for entire utterances in L1 containing L3 words, which are not pre-subtitled. For instance, in Episode 7, the utterance, “Am I gonna be a tía soon?” is audio subtitled with, “Am I going to be an aunt soon?”.

Another strategy found in the AD of the show, albeit less frequently than selective reproduction, is vehicular matching, when no AST is added, and the audience hears the original utterance(s) in L3. Finally, for what concerns L3 songs, they are not translated in the audio-described version, which reflects the absence of pre-subtitles in the non-audio-described version.

3.4. 1899

1899 consists of a single season of eight episodes, which last from 50 to 62 min. The TV show is the most multilingual in the sample, both in terms of number of languages and in terms of L3 presence, which occupies around 70% of all dialogues.

As far as SDH is concerned, in the eight episodes of 1899, the most common strategy is translation plus explicit attribution, which is used for the vast majority of L3 utterances (see Example 20).

Example 20.

 

[in Polish] I’m sorry.

I am very sorry, sir.

I know I am not allowed in here./I was just having a break.

[in French] Stay.

[in Polish] I will go now.

[in French] Stay.

I don’t want to hurt you. I’m hungry.

(Episode 1)

This choice of strategy is far from surprising. Indeed, the original version of the show includes part-subtitles for virtually all instances of L3, which are necessary for the primary target audience to understand the content of the numerous dialogues in L3, as well as to follow the plot. There are also some cases of diegetic interpreting in the original dialogue. For instance, in Episode 4, a sailor is speaking in German to some Danish passengers who do not understand his language, except for two of them, who translate for the others. In this case, the subtitles use translation plus explicit attribution instead of opting for vehicular matching or explicit attribution (Example 21).

Example 21.

 

[in German] Wait.

There is something you need to know.

The morning we returned/from the Prometheus

we received a message from…

from the ship company.

[in Danish] Two days ago,/they got a message from the ship company.

[in German] It read, “Sink ship.”

[in Danish] It said/we should sink the ship we found.

In Episode 6, a Polish character and a Chinese girl speak their own language and struggle to understand what the other says, so they translate some words and short phrases into English, a language that the Chinese girl is only slightly familiar with. The closed captions use translation plus explicit attribution in this case as well (Example 22).

Example 22.

 

[in Cantonese] Is this yours?

Is this from a girl?

[in English] Girl?

No.

No girl.

[in Polish] My brother.

Vehicular matching is found only once, for an expression in French that is transcribed in italics in the subtitle without translating it (“Et voilà”, Episode 3). Explicit attribution is used only twice, specifically for two songs in L3, whose lyrics are neither translated nor transcribed in the subtitles but simply introduced by a subtitle indicating the type of song and the language: “[Child singing German folk song]” and “[German folk song]”. Linguistic homogenisation is found only once for a French utterance which is translated in the subtitle without indicating the change of code. Moreover, in three instances, L1 (i.e., English) is not labelled, which may create confusion for the viewer, as the previous utterance was spoken in a different language. Regarding the presence of accents, no indication is given (e.g., English with a Spanish, German, or French accent) in the subtitles.

For what concerns AD, 1899 constitutes a totally different case in the sample. In fact, the strategy used is homogenising convention, which means that all the utterances in L3 are fully translated by means of dubbing, thus erasing multilingualism altogether and producing a monolingual text. No further information is provided in the English AD, so the blind and partially sighted audience will not be aware at all of the presence of L3. It should be noted that in the case of this show, Netflix offers two different audio tracks in English: one is labelled “English [Original]” and the other “English–Dubbed”. The AD track is associated with the latter, possibly because the original text would present many difficulties for AD and AST. It would be very challenging, for example, to opt for selective reproduction plus language information, as the constant alternation of languages would make it difficult to include information on the languages spoken. Perhaps in a case such as this, in order not to completely erase multilingualism, AST with a voice-over effect (i.e., selective reproduction) or AST with a dubbing effect after the L3 is announced by the audio describer (i.e., explicit attribution) could be used.

4. Discussion and Concluding Remarks

This paper has examined how the presence of L3 has been made accessible for deaf and hard-of-hearing as well as blind and partially sighted audiences in four recent multilingual TV shows available on Netflix, taking into consideration the strategies proposed by Szarkowska et al. (2013, 2014) for SDH and Iturregui-Gallardo (2020) for AD/AST. Although Netflix provides some guidelines for the treatment of L3 in SDH and AD, the results have shown differences from one TV show to another in the strategies adopted in both practices.

Regarding SDH, in Emily in Paris, the main strategy is translation plus explicit attribution, which is applied consistently whenever part-subtitles are provided in the original version. However, when part-subtitles are absent in the original version, different strategies have been found, namely translation plus explicit attribution, vehicular matching, explicit attribution, and linguistic homogenisation. In From Scratch, three main strategies have been observed in the closed captions: translation plus explicit attribution, vehicular matching, and explicit attribution. The last two are mainly found when part-subtitles are not provided in the original version. There are no cases of linguistic homogenisation. The two main strategies in Gentefied are translation plus explicit attribution and linguistic homogenisation, followed by vehicular matching, and to a lesser degree, by explicit attribution. The latter two strategies are found most notably when L3 utterances are not pre-subtitled. In 1899, the largely preponderant strategy is translation plus explicit attribution, while explicit attribution, linguistic homogenisation, and vehicular matching are used only sparingly.

A common element in the closed captions of the shows, except for 1899, is the combined use of translation and vehicular matching, which creates subtitles characterised by code-mixing. This approach contributes to making L3 more visible while at the same time providing access to content. There are, moreover, similarities in the way the four shows deal with accents, which are not marked in the closed captions despite Netflix’s recommendation in the guidelines, thus neutralising this aspect of multilingualism. Moreover, the treatment of L3 songs is quite consistent across the shows. Overall, songs in L3 are neither translated nor transcribed but only labelled, except for songs that are part of the diegesis in Emily in Paris, which are translated into L1, thus respecting the presence of part-subtitles. However, there is one case in Gentefied where a song sung by a character is translated in closed captions despite not being subtitled in the original version. On-screen text in L3 is also accounted for in the closed captions of the four shows through the strategy of translation plus explicit attribution.

In terms of AD, with the exception of 1899, where the strategy of homogenising convention is applied for the entire show—thus erasing all traces of multilingualism—the two main strategies in the other three shows analysed are selective reproduction and vehicular matching, which are mostly chosen according to the presence or absence of part-subtitles. The use of these two strategies means that L3 is never explicitly referred to in the audio-described shows, requiring the audience to infer this information for themselves.

Although the choice of the main strategies to deal with L3 is a common element in all three audio-described shows, there are some significant differences in the way audio subtitles are introduced and voiced. Firstly, when introducing AST, the word “subtitle(s)” is read quite consistently in Emily in Paris, yet less frequently in Gentefied, and never in From Scratch. This discrepancy seems to contradict the guidelines provided by Netflix, which recommend stating “subtitles” when necessary to avoid confusion. However, according to Szarkowska and Jankowska (2015a, p. 214), not announcing the appearance of subtitles “can be considered less disruptive for viewers, who are not constantly reminded that they are watching a subtitled film”. Secondly, the name of the character speaking is specified whenever necessary in Emily in Paris, while it is never mentioned in From Scratch and Gentefied. Thirdly, regarding the number of voices and their gender, no specific pattern could be identified. In Emily in Paris, besides the voice reading the AD script, two different voices, one female and one male, read the audio subtitles. The same happens in From Scratch. On the other hand, in Gentefied only one voice (female) reads both the AD script and the AST. These different approaches can have consequences in understanding L3 dialogue. For example, not mentioning the character’s name may be confusing for the audience, especially in fast exchanges and where there are multiple speakers.

Another interesting difference concerns the use of selective reproduction, which is normally applied when part-subtitles are provided in the non-audio-described version. However, the analysis has shown that in Gentefied this strategy is sometimes applied when single L3 words or expressions are inserted in the discourse in L1. Audio subtitles are thus added in the audio-described version for utterances characterised by code-mixing. This choice is difficult to justify; since the audience of the non-audio-described version does not have access to the translation of such words or expressions, it seems strange that the blind and partially sighted audience should be given this explanation.

Another noteworthy aspect is the treatment of L3 songs: the results have shown that foreign songs are not translated in AD when they are part of the shows’ soundtrack and are not pre-subtitled. The blind and partially sighted audience will thus hear the original song, although some parts may be obscured by the voice reading the AD script. A different case is found in Emily in Paris; when songs are pre-subtitled because they are part of the narrative and it is a character who sings, audio subtitles are added in the audio-described version. As noted by Fryer (2016, p. 122), “research is needed to discover ways in which lyric content of foreign songs can be conveyed without masking the music”, and this is even more challenging “when a song is used as background music with competing dialogue over the top”.

On the whole, considering the strategies used in both SDH and AD, it can be seen that the presence of L3 is addressed in different ways, not only from one TV show to another but also within the same show, thus leading to different effects, which range from neutralisation to L3 visibility. As pointed out by Corrius et al. (2019) in their study of five multilingual films audio described in Spanish, it is not surprising to find differences in AD practices, especially in countries that lack a tradition of AD. The same could be said for SDH. However, one wonders whether the same consideration might apply to English AD and SDH, which both have a long-standing tradition in these fields.

The preliminary results of this study need to be compared with the analysis of additional TV series as well as of different audiovisual products (e.g., films, documentaries, reality shows, etc.) available on different platforms in order to obtain more extensive data. Furthermore, it would be important to investigate through reception studies which preferences and expectations deaf and hard-of-hearing and blind and partially sighted audiences actually have in terms of multilingualism.

As this area of SDH and AD merits more investigation, this study hopes to encourage further research into the accessibility of multilingual audiovisual texts and the treatment of L3 for deaf and hard-of-hearing, as well as blind and partially sighted audiences.