Regulation and Control: What Bimodal Bilingualism Reveals about Learning and Juggling Two Languages

Abstract

In individuals who know more than one language, the languages are always active to some degree. This has consequences for language processing, but bilinguals rarely make mistakes in language selection. A prevailing explanation is that bilingualism is supported by strong cognitive control abilities, developed through long-term practice with managing multiple languages and spilling over into more general executive functions. However, not all bilinguals are the same, and not all contexts for bilingualism provide the same support for control and regulation abilities. This paper reviews research on hearing sign–speech bimodal bilinguals who have a unique ability to use and comprehend their two languages at the same time. We discuss the role of this research in re-examining the role of cognitive control in bilingual language regulation, focusing on how results from bimodal bilingualism research relate to recent findings emphasizing the correlation of control abilities with a bilingual’s contexts of language use. Most bimodal bilingualism research has involved individuals in highly English-dominant language contexts. We offer a critical examination of how existing bimodal bilingualism findings have been interpreted, discuss the value of broadening the scope of this research and identify long-standing questions about bilingualism and L2 learning which might benefit from this perspective.

1. Introduction

Although bilingualism takes many different forms, there is evidence that knowing and using more than one language leads to both behavioral and neurocognitive changes or accommodations (Bialystok 2017; Pliatsikas 2020). Bilinguals and multilinguals experience language co-activation because all of their languages come online to some degree when even one language alone is processed (Kroll et al. 2008). The co-activation of the bilingual’s two languages occurs at all levels of language processing (see Kroll et al. 2015 for a review). In this context, bilinguals must be able to regulate the intended language so that the unintended language does not intrude. How speakers acquire and use mechanisms of control to enable proficient bilingual performance is a topic that has been at the center of current research on bilingual language processing (e.g., Declerck and Koch 2022). The same mechanisms of regulation and control have been shown to be important during language learning.

In this paper, we examine the way that bilinguals who use markedly different languages, one a spoken language and the other a signed language, regulate the use of their two languages to enable skilled language use. We first review what is known about the ways that unimodal bilinguals who speak two languages engage these processes, and we then compare the evidence on unimodal bilinguals to bimodal bilinguals. We note at the outset that bimodal bilingualism is not the only form of bilingualism in which the two languages differ in significant ways. Many studies of unimodal bilinguals have examined language pairings with distinct lexical, grammatical, and phonological features. What is striking about the research on cross-language activation and its consequences for regulation and control is that the findings are largely similar across many different pairs of languages. There are modulations that result from language distance, but the overarching picture is one of greater similarity than the difference (Degani et al. 2018; see also Barac and Bialystok 2012). Critically, the role of contextual factors has been shown to be as or more important than language distance (e.g., Beatty-Martínez et al. 2020 found effects of contextual factors in Spanish–English bilinguals that are comparable to those found in Chinese–English bilinguals by Zhang et al. 2015, 2021). The environment places differential demands on speakers as a function of the opportunities to use the two languages across different contexts and distinct social networks (e.g., Green and Abutalebi 2013; Gullifer and Titone 2020). The form of bilingualism may itself vary with these demands or be independent of them.

In what follows, we review the research that has laid a foundation for understanding the regulation and control of a bilingual’s two languages, and we then examine bimodal bilingualism in more detail. We view bimodal bilingualism as a phenomenon of interest in and of itself but also as a tool to investigate the constraints and plasticity associated with the mappings between language, cognition, and their neural underpinnings.

2. What Is Bimodal Bilingualism?

Bimodal bilinguals can be Deaf individuals who know a natural signed language and the spoken and/or written form of a spoken language (depending on their language experience and when they became deaf), or hearing individuals who know a spoken language and a signed language (see Berent 2004 for an overview of the forms that bimodal bilingualism can take).1 Here, we restrict our focus to hearing bimodal bilinguals (see Morford et al. 2011, 2014, 2019 for studies of Deaf sign-print bilinguals). Crucially, bimodal bilinguals’ languages are in two different modalities.2 This offers an opportunity to explore the role of overlapping forms on different aspects of bilingual linguistic and cognitive behavior (e.g., Morford and Kroll 2021). For example, what does language co-activation look like when there is no shared phonology? Does the way that bilinguals regulate their languages differ, quantitatively or qualitatively, when the two languages do not rely on the same articulatory and perceptual systems, and what are the consequences of this for non-linguistic cognitive processing?

In many ways, bimodal bilinguals are similar to unimodal bilinguals. Children of Deaf Adults (CODAs), that is, hearing individuals who grew up in deaf families, in particular, have much in common with those unimodal bilinguals who use their two languages in different contexts and cultures (e.g., Heritage speakers of a home language who speak the language of the community at school and work but the use the home language with family).3 Natural sign languages are phonologically, lexically, and grammatically different from the spoken languages surrounding them (Sandler and Lillo-Martin 2006). That means that bimodal bilinguals learn two distinct languages, and many also come to know two different cultures (Singleton and Tittle 2000). Similarly, although the two languages are in different modalities, they appear to be supported largely by the same neural systems, much as is observed for unimodal bilinguals. This is the case when comparing native sign language and spoken language in different individuals (see MacSweeney et al. 2008; and Emmorey 2002, for reviews) and in bimodal bilinguals (Emmorey et al. 2014; see Emmorey et al. 2016, for an overview).

Despite many similarities between unimodal and bimodal bilinguals, a crucial difference is that bimodal bilinguals can use and comprehend both of their languages at the same time because they can sign while they speak or speak while they sign. It is important to note that this does not mean that bimodal bilinguals regularly converse fluently and grammatically in both languages at the same time; rather, bimodal bilinguals often accompany some spoken words in an utterance with signs, or vice versa. Nevertheless, this ability means that bimodal bilinguals are not subject to the constraints that unimodal bilinguals face of being physiologically unable to produce their two languages at the same time because their languages are perceived and produced using different production and perceptual systems. In contrast, spoken languages are primarily produced orally and perceived aurally, whereas signed languages are produced manually and with the body and perceived visually.4 While language processing in both modalities largely makes use of the same neural networks, some differences have been reported in patterns of brain activity responsible for modality-specific language components, e.g., listening vs. watching, as well as differences in motor regions moving hands vs. vocal tract (Corina et al. 2013).

Using both languages at the same time is not something that is merely technically possible. Unlike unimodal bilinguals who know two spoken languages and who often switch languages sequentially, a phenomenon known as code-switching, bimodal bilinguals tend to code-blend, that is, to use two languages at the same time. Such language blending is a normal and natural process for bimodal bilinguals (Emmorey et al. 2008a), and especially CODAs (Bishop 2010; Bishop and Hicks 2005; Zeshan and Panda 2018; Kuntze 2000). 5 Code-blending is not restricted to lexical items (e.g., signing HOME in ASL while saying home in English). A study by Pyers and Emmorey (2008) showed that bimodal bilinguals use facial expressions that convey grammatical information in ASL while speaking in English. In a communicative task in English, bimodal bilinguals produced more ASL-like facial expressions (i.e., eyebrow raises to indicate if-statements and furrowed brows to indicate wh-questions) than did non-signers, despite the fact that their interlocutor did not know ASL. Language blending is not unique to proficient bimodal bilinguals. There is a growing body of evidence from studies of adult sign language learners, suggesting that signs or sign language-based distinctions begin appearing in the co-speech gestures that accompany the spoken language from very early stages of language acquisition (Casey et al. 2012; Gu et al. 2019; Weisberg et al. 2020; Frederiksen 2021).

Bimodal bilinguals’ ability to use signed and spoken language at the same time has consequences on how they manage their languages, which in turn affects how bilingualism modulates cognitive control abilities in this population. Understanding how two languages are managed under these conditions is not only informative in its own right but also offers a way of establishing how different aspects of the bilingual experience are related to cognitive abilities and processes.

Language Co-Activation in Bimodal Bilinguals

While bimodal bilingualism presents some unique opportunities for using the two languages together, it also shares a fundamental feature of unimodal bilingualism: When one language is presented alone, the other language is activated to some degree. An extensive body of research on unimodal bilingualism has established the parallel activation of the two languages when unimodal bilinguals listen to speech, when they read, and when they plan speech in each of the two languages. Reviews of the evidence on the nonselective activation of bilinguals’ two languages are widely available (e.g., Dijkstra and Van Heuven 2018; Kroll et al. 2006, 2015; Kroll and Navarro-Torres 2018). An important observation in this research on unimodal bilingualism is that bilingualism with spoken languages does not depend on shared properties of the written or spoken form of the language; even languages that do not share the same written script or phonological form give rise to cross-language activation (e.g., Hoshino and Kroll 2008; Thierry and Wu 2007; but see also Costa et al. 2017 for an account that assumes mapping between L1 and L2 during learning instead of cross-language activation during processing). These findings are important because they suggest that we might also see cross-language activation in bimodal bilinguals and that bimodal bilinguals, like unimodal bilinguals, may need to engage regulatory mechanisms that draw on domain-general cognition.

Several studies have shown that although bimodal bilinguals’ languages make use of two different modalities, these bilinguals experience the same kind of language co-activation as unimodal bilinguals do (Shook and Marian 2012; Giezen et al. 2015; Giezen and Emmorey 2016; Villameriel et al. 2016; Williams and Newman 2016). Morford et al. (2014) investigated if hearing English speakers who were proficient signers of American Sign Language (ASL) would activate ASL while performing a lexical decision task in written English. The English-ASL bilinguals were presented with English word pairs that were semantically related (heart-brain) or unrelated (baby-lion), and their task was to indicate with a button press whether the words were related in meaning or not. Unbeknownst to the participants, half of the words in each semantic condition had ASL translation equivalents that were phonologically related; that is, they shared some combination of hand shape, movement, or location in signing space (i.e., shared hand shape and movement in the semantically unrelated, but phonologically related ASL word pair MOVIE-PAPER in Figure 1).

Despite there being no phonological overlap between ASL and English because the languages are produced in different modalities, Morford and colleagues found that the English-ASL bilinguals co-activated ASL while performing the task in English. Specifically, participants were slower to indicate that semantically unrelated English words were not similar in meaning when the ASL translation equivalents were related in phonology, compared to when semantically unrelated words were also phonologically unrelated in ASL.

These findings highlight that language co-activation is not restricted to language pairs with shared form elements and suggest that it is instead a feature of the bilingual language system. Bimodal bilinguals, however, are uninhibited by the factors that restrict unimodal bilinguals to using one language at a time, and code-blending may thus reflect how the bilingual language system handles both languages being active in the absence of any articulatory constraints. The next sections present evidence on the bilingual adaptations resulting from the need to regulate language co-activation and discuss how the fact that bimodal bilinguals may have a much-reduced need to regulate this offers a way to investigate the role of language regulation in the development of bilingual cognitive abilities.

3. Control and Regulation in Bilinguals

The traditional approach to bilingualism research, perhaps especially in studies taking a cognitive perspective on bilingualism, has aimed to identify group differences in bilinguals compared to monolinguals across different cognitive dimensions. Understanding that the bilingual is not two monolinguals in one (Grosjean 1989), research has sought to characterize the nature of the difference between monolinguals and bilinguals, not only in terms of language use but also in the cognitive and neural consequences that result from the use of more than one language. An obvious difference between monolinguals and bilinguals is that bilinguals have two languages and must always select which one to use, which creates different demands on language monitoring in these two groups of language users, especially because of the demands in the bilinguals on the mechanisms responsible for language selection. In this way, cross-language activation, and the need to regulate it, results in different every-day cognitive demands for bilinguals compared to monolinguals and different bilingual adaptations to meet these demands (e.g., Bialystok 2017). While there is considerable ongoing controversy about the scope of the consequences and adaptations produced by bilingual experience (see, for example, Antoniou 2019; van den Noort et al. 2019; and Grundy 2020), the fact that bilinguals’ two languages are always active is uncontroversial. Yet, the nature of cross-language activation and its regulation is also complex as research uncovers more details about how variation in the bilingual experience modulates aspects of bilingual language and cognition. Although variation in factors such as the age of acquisition and proficiency level has long been of interest, other types of variation have received limited attention until recently. If we consider how people around the world come to learn more than one language, it becomes clear that there are many different circumstances that lead to bilingualism ((Baum and Titone 2014; Fricke et al. 2019), just as there are different environments in which bilinguals use their languages when they have already been acquired (Luk and Bialystok 2013; Surrain and Luk 2019)). Bimodal vs. unimodal bilingualism represents an obvious instantiation of variability in bilingual experiences. As discussed, bimodal bilingualism is in some respects drastically different from unimodal bilingualism while also sharing many characteristics with it. This offers a unique opportunity to explore how different variable aspects of bilingualism influence bilingual cognitive adaptations. In recent years, a growing interest in differences among bilingual populations has begun to uncover significant variation in what language co-activation looks like based on individual circumstances and in how bilinguals’ language regulation needs and abilities change in response (Green and Abutalebi 2013; Giezen et al. 2015; Zirnstein et al. 2018; Treffers-Daller 2019; Beatty-Martínez et al. 2020). From this perspective, the properties of bimodal bilingualism represent variation not only in the modality difference compared to unimodal bilingualism but also among bimodal bilinguals in different circumstances.

In what follows, we review foundational evidence about language regulation and control from studies of unimodal bilingualism and then discuss how accounting for variation in bilingual experiences, including in modality, reveals systematicity in previous findings.

3.1. Cognitive Control Adaptations

Cognitive control refers to domain-general abilities that help with regulating mental activity in order to resolve information conflicts (Braver 2012; Miyake and Friedman 2012). While cognitive control is a domain-general ability, it has important uses in language processing, for example, in resolving competing interpretations of sentence or utterance meaning (Novick et al. 2005; Nozari et al. 2016; Hsu et al. 2017). Similarly, dynamic engagement of cognitive control has been shown to facilitate efficient language processing (Hsu and Novick 2016; Hsu et al. 2021; Ovans et al. 2022).

Many studies have shown that bilingualism can lead to behavioral changes, not only in language use but also in the ways that cognitive and neural resources are recruited to enable fluent language use. Proficient bilinguals often outperform monolinguals on a variety of tasks measuring executive functions, including cognitive control (Bialystok et al. 2008; Bialystok and Craik 2022). This is assumed to be due to bilinguals’ long-term experience with managing two languages. Specifically, the extensive experience in engaging in mental behaviors involved in controlling one’s languages is assumed to confer benefits to other non-verbal domains (Bialystok 2009). Bilingualism also leads to changes to neural patterns in the brain, as well as changes to anatomical structure, including increased grey matter and cortical thickness in the temporo-parietal cortex, as well as enhanced white matter integrity, e.g., in the inferior fronto-occipital fasciculus (Li et al. 2014; Abutalebi and Green 2016), some of which have been linked to observed behavioral changes to cognitive control (Della Rosa et al. 2013). Such neural and behavioral changes can have long-term consequences, as evidenced, for example, by research showing that bilingualism acts as a protective factor against dementia, such that bilinguals experience a later onset of symptoms relative to monolinguals (Bialystok et al. 2007).

Studies in the past decade have begun to identify the ways in which variability in bilinguals’ language and learning experiences may affect cognitive processes. According to the adaptive control hypothesis (Green and Abutalebi 2013), the interactional contexts in which bilinguals live and use their two languages are predicted to differentially shape control processes. For example, this hypothesis holds that bilinguals who generally use their languages separately in different environments are expected to adapt their control processes to their surroundings differently than bilinguals who generally use both languages in many contexts but with different speakers, and both of these bilingual groups are expected to differ from those regularly code-switch, that is, who use both languages in many contexts and with the same speakers (Green and Abutalebi 2013).

Investigating bimodal bilinguals has been particularly fruitful for disentangling the possible contributing factors underlying changes to cognitive control in bilinguals. Bimodal bilinguals’ ability to use both languages at the same time suggests that they have a reduced need to fully inhibit one language while they are using the other. If processes such as cognitive control develop in response to the need to manage one’s languages, then bimodal bilinguals may not necessarily show the same kinds of changes to cognitive control abilities that unimodal bilinguals do. Indeed, Emmorey et al. (2008b) found that adult unimodal but not bimodal bilinguals outperformed monolinguals on the speed of decision on a set of tasks measuring cognitive control. They argue that this provides evidence that the enhanced cognitive control often observed in bilinguals can be traced to the experience of managing two languages in the same modality rather than simply managing the representation of two languages. Similarly, Olulade et al. (2016) measured differences in grey matter volume in frontal brain areas associated with executive functions and found that while unimodal bilinguals had larger grey matter volume compared to monolinguals, bimodal bilinguals did not (but see Zou et al. 2012b). What these studies suggest is that differences in cognitive control do not necessarily happen as a result of bilingualism, per se. In this way, research on bimodal bilinguals offers the chance to establish causal links between bilingual experiences and outcomes. We further note that the interest in bimodal bilingualism has the potential to uniquely contribute to the paradigm shift in the broader field of bilingualism over the past decade, away from the traditional, relatively narrow interest in how bilingualism, broadly construed, differs from monolingualism and towards an interest also in individual differences between bilinguals as well as differences between bilingual groups whose language experiences differ in ways that go beyond differences in proficiency.

Cognitive Control and Language Learning

In proficient bilinguals, cognitive control abilities and the ability to manage multiple languages are assumed to develop alongside language proficiency. Results from studies with babies and toddlers have suggested that bilingualism may lead to different cognitive development with respect to memory generalization and improved cognitive control abilities even before the age at which most children master basic two-word utterances (Brito and Barr 2012) or even start to produce words (Kovács and Mehler 2009; Ferjan Ramírez et al. 2017). More uncertain is what role cognitive control plays for developing bilinguals, that is, learners of a second language (L2) after childhood when the first language (L1) has been solidly established, with robust links between world and language, such as between concepts and lexical items, and what kind of relationship exists between cognitive control, language regulation and L2 proficiency and use at different stages of acquisition.

For decades, second language learning research has investigated what enables adult learners to become proficient in an L2, investigating both external and internal factors (e.g., learning environment vs. motivation, Luque and Morgan-Short 2021). An existing assumption is that the skills and processes that enable the proficient bilingual to function in their two languages are also related to achieving proficiency in a second language in the first place (Michael and Gollan 2005). One possibility is that individual differences in cognitive control abilities in monolinguals predict their success with L2 learning. In order to assess whether increased cognitive control helps initial language learning, studies have compared monolinguals and bilinguals on tasks related to novel word learning (Cenoz 2003; Kaushanskaya and Marian 2009). A study by Bartolotti and Marian (2012) that specifically investigated word retrieval in a newly-learned language found that resolution of between-language competition occurred earlier in bilinguals than in monolinguals. One interpretation of these results is that individuals with better cognitive control abilities make more efficient L2 learners. Another possibility is that cognitive control changes as L2 proficiency and/or the need to regulate activation of the L1 vs. L2 increases. Luque and Morgan-Short (2021) examined the relationship between L2 proficiency and different cognitive measures, including cognitive control. They administered a range of cognitive and L2 proficiency tasks to test the correlation between cognitive control and L2 proficiency. Reactive and proactive cognitive control was measured using the AX Continuous Performance Task (AX-CPT), in which participants’ reaction times are measured to five-letter strings in which the first letter is the cue and the final letter is the probe (see Braver 2012 for a discussion of how this task provides a framework for analyzing cognitive control). The study found a negative correlation between L2 proficiency and overall processing time and reactive control, with the more proficient L2 users reacting faster overall as well as on trials measuring reactive control. Across studies, however, results have been mixed, with a relationship between cognitive control and second language learning occurring only sometimes and varying as a function of proficiency and task (Luk et al. 2011; Bak et al. 2014; Misra et al. 2012; Vega-Mendoza et al. 2015; Xie 2018).

In this context, studying bimodal bilingualism might offer the means to disentangle the relationship between L2 learning and cognitive control. As discussed above, findings from adults who learned a sign language and a spoken language in childhood have suggested that their cognitive control abilities may not show the same enhancements as in unimodal bilinguals, and this difference is explained by the fact that bimodal bilinguals are not required to regulate their two languages as carefully as unimodal bilinguals. If proficient bilinguals do not require the kind of cognitive control abilities that unimodal bilinguals do to function in their two languages, then learning a sign language as a second language may similarly result in little or no changes to cognitive control, nor should individual differences in cognitive control predict success in learning a signed L2. To date, no studies that we are aware of have examined the effect of cognitive control on the acquisition of a signed language, nor whether knowing a second sign language facilitates learning a new spoken language, as in the study by Bartolotti and Marian (2012). Interestingly, however, the results from two studies examining the learning of a sign language as a second language in adulthood after a childhood of only spoken language exposure suggest that cognitive functions, in fact, may change in adult sign language learners. In a longitudinal study of ASL interpreting students, Macnamara and Conway (2014) found improvement in emerging bimodal bilinguals’ cognitive abilities, suggesting that, under the right circumstances, bimodal bilingualism can, in fact, have similar effects as unimodal bilingualism, although it remains unclear whether the level of improvement observed in the study would be enough to differentiate the bimodal bilinguals from monolinguals, as well as how much the cognitive changes should be attributed to specifically to interpreter training as opposed L2 learning in and of itself. Similarly, changes to cognition may appear in bimodal bilinguals who learned sign language after childhood. Proficient bilinguals who have learned both speaking and signing in childhood do not show increased grey matter volume in brain areas such as the dorsolateral prefrontal and parietal cortices that are implicated in executive functions such as attention shifting, inhibition, and conflict detection and resolution (Derrfuss et al. 2004; Cole and Schneider 2007) and that are frequently seen to have greater volume in proficient unimodal bilinguals (Olulade et al. 2016). However, Chinese speakers who learned Chinese Sign Language as an L2 in adulthood showed structural differences from monolinguals in specific brain areas implicated in language switching and control (Zou et al. 2012b, see also Li et al. 2015).

In sum, many questions remain for future research about the relationship between cognitive control and L2 learning for bimodal bilinguals and how this may differ from proficient bimodal bilinguals. For proficient bilinguals, there is behavioral and neurocognitive evidence to support the conclusion that the changes often observed in unimodal bilinguals’ cognitive control abilities result from the experience of needing to regulate two languages in the same modality, and as bimodal bilinguals need less regulation, they may not show similar cognitive enhancement. However, the evidence from signed L2 learning, as well as the research focused on bilingual variation in unimodal bilinguals, makes it increasingly clear that differing patterns of bilingual language use may contribute to the variation observed in bilingual cognition, including cognitive control patterns (Treffers-Daller 2019; Green and Abutalebi 2013). The next section discusses questions related specifically to language regulation in bilinguals, how cognitive control is modulated by variation in how much language regulation a bilingual engages in as a function of language environment and interactional contexts (Zirnstein et al. 2018; Beatty-Martínez et al. 2020), and how research on bimodal bilinguals may shed light on these issues.

3.2. Language Regulation

Language regulation, a term first used by Zirnstein et al. (2018), is closely related to but not fully overlapping with cognitive control. The term describes bilinguals’ use of different executive functions to shift the activation state of the L1 and L2, such that the demands of the environment can be met (Kroll et al. 2022). Bilinguals must learn flexibility in how they inhibit and disinhibit their two languages as a function of a range of factors, including immersion, language status, individual-level proficiency, conversational partner, and other types of contexts. Language regulation is also important for learning an additional language in the first place. Findings from studies of adult language learners have suggested that successfully learning a second language partially depends on the ability to regulate the first (or native) language. While language regulation likely involves cognitive control, recent findings suggest that the two processes do not fully overlap (Kang et al. 2020; Zirnstein et al. 2018). This section discusses evidence of language regulation stemming from language switching as well as language learning and immersion and also discusses the relationship between cognitive control, language regulation, and a bilingual’s interactional contexts, emphasizing what research on bimodal bilingualism contributes to current understandings of these phenomena.

3.2.1. Language Switching as Evidence of Language Control and Regulation

Much of what is known about language regulation comes from studies of language switching. The critical insights from this work are that although bilinguals can communicate in both their languages and many proficient bilinguals code-switch frequently, it can be costly to switch between languages. Especially, the type of cued or forced switching employed in typical experimental paradigms results in high switching costs (see Zhu et al. 2022). A foundational study by Meuter and Allport (1999) used a cued language switching paradigm in a number naming task and showed that naming was overall faster in L1 as compared to L2. Crucially, on trials where bilinguals had to switch language (switch trials), response latencies were longer than on trials where the same language was used in the preceding trial (non-switch trials) compared within the same language. This difference is termed the switching cost. Meuter and Allport moreover found that switch costs were asymmetrical between the L1 and L2, with the L2 to L1 switch producing longer response latencies than L1 to L2 switches, and, importantly, L1 naming becoming slower than L2 naming on switch trials, suggesting that it is more effortful to regulate the L1 compared to the L2. Switch costs have been found both in production (e.g., Costa and Santesteban 2004; Meuter and Allport 1999) and comprehension (e.g., Grainger and Beauvillain 1987; Thomas and Allport 2000). Based on these insights, the field has long assumed that language switching is costly and that the cost associated with language switching results from the effort required to regulate the relative activation of the two languages, specifically as related to overcoming the inhibition from the preceding trial (e.g., Kroll et al. 2008; Philipp and Koch 2009).

Cued language switching has also been investigated for bimodal bilinguals. As research has shown that both languages are similarly activated in bimodal bilinguals as in unimodal bilinguals (e.g., Shook and Marian 2012), bimodal bilinguals should be similar to unimodal bilinguals in needing to rely on language regulation in order to switch between languages. Dias et al. (2017) investigated language switch costs in bimodal Spanish-Spanish Sign Language (LSE) bilinguals and found that they performed similarly to unimodal bilinguals in exhibiting greater switch costs when switching from the weaker (L2) to the stronger (L1) language. However, in a study comparing language switch costs in German–English-German Sign Language (DGS) trilinguals, Kaufmann et al. (2018) found that although there was a cost to switching between German and DGS, the costs were higher in unimodal German–English contexts than bimodal German-DGS contexts in intermediate sign language learners. One suggested explanation is that the difference is due to the fact that it is the lexical item that has to be inhibited in the unimodal context, but in the bimodal context, only the articulators need to be inhibited—and that lexical inhibition is costlier.6 Casey et al. (2012) similarly propose that language control could be uniquely challenged when the two languages are in different modalities. This raises the question of whether language regulation is, in fact, the same unified process in the contexts of unimodal vs. bimodal bilinguals.

3.2.2. The Locus of Language Switch Costs

Emmorey et al. (2008a) conducted one of the first studies to highlight the unique capacity of bimodal bilinguals to use both of their languages simultaneously. This work demonstrated that bimodal bilinguals (CODAs) tend to code-blend rather than code-switch the way unimodals do. Accordingly, research on bimodal bilinguals has provided a way to tease apart which aspect of switching is responsible for the cognitive cost. Several processes are involved in switching from one language to the other. Simplistically, for an individual to switch between languages, they first must recognize that a switch has to be executed, then they must inhibit (“turn off”) one language and activate or increase activation (“turn on”) of the other language. While there is broad agreement that language switching is costly, it is less clear which components of the switch are responsible for the cost. Some theories have emphasized activation as being costly, while others have emphasized inhibition (see Declerck and Philipp 2015, for review). In unimodal bilinguals, the switching on and off happens at the same time; that is, in order to produce a language switch, the unimodal bilingual must simultaneously turn off one language and turn on the other. In bimodal bilinguals, these two processes can be separated by taking advantage of bimodal bilinguals’ ability to use both languages at the same time. Specifically, studies of switch costs in bimodal bilinguals include dual response trials, that is, trials in which items are named in both languages at the same time by signing and speaking (see Kaufmann et al. 2018; and Emmorey et al. 2020). When bimodal bilinguals name in a single language after naming in both languages simultaneously, this isolates the process of turning a language off. Conversely, naming in both languages simultaneously after naming in only one of the languages isolates the process of turning a language on. In this way, preceding and following dual response trials with single language trials allows for an investigation of whether costs for turning a language on (going from single language trial to dual language trial) differ from the costs for turning a language off (going from dual language response to single language).

A study by Emmorey et al. (2020) compared response latencies in a task in which bimodal bilinguals named pictures in English, ASL, and both languages simultaneously. Their results showed no cost to bimodal bilinguals for switching a language on, only for switching one-off. These findings about behavior have been corroborated by neuro-imaging research. Recording neural activity using magnetoencephalography (MEG) while bimodal bilinguals performed a picture naming task, Blanco-Elorrieta et al. (2018) showed that the brain regions involved in cognitive control are not involved when bimodal bilinguals engage in a language, only when they inhibit one. Studying bimodal bilinguals thus offers a way to test hypotheses about links between different bilingual language behaviors and cognitive processes, which can be assumed but not tested from correlational studies of unimodal bilinguals.

3.2.3. What Code-Blending Reveals about the Bilingual Language System

The studies discussed above—showing that inhibition and not activation of a language is what is costly—help explain why bimodal bilinguals code-blend when intuition suggests that using two languages at the same time should be more effortful than using one. If adding a language is not costly, however, using two languages at the same time may, in fact, not be more effortful than using only one. Emmorey et al. (2008a) demonstrated that bimodal bilinguals frequently express the same lexical meaning in their two languages simultaneously instead of suppressing one of the languages. Along the lines of the discussion in the previous section, this suggests that it may be less costly to select two lexical representations compared to selecting one and suppressing the other (p. 57). Emmorey et al. (2012) provided empirical evidence for this suggestion by investigating the processing of code-blends (see also Giezen and Emmorey 2016; Emmorey et al. 2020). They showed that production and comprehension of simultaneous ASL and English (i.e., signing CUP while saying ‘cup’) do not appear to be more difficult than using one language at a time. Specifically, in production, ASL signs were produced with the same speed in code-blends compared to ASL alone. English words were slower in code-blends, but this was attributed to the desire to synchronize lexical onsets in the two languages—with ASL being overall slower (even in monolinguals)—and thus not attributable to processing costs. In comprehension, code-blends facilitated access to both languages. The results suggest that the dual language mode is not more costly than the single language mode, a remarkable finding given studies of non-language dual response tasks show worse performance in the dual compared to a single task mode (see Pashler 1994, for review). While simultaneously naming in two languages at the same time is less effortful than naming in one language while turning the other language off, the relative cost of inhibiting one language compared to dual lexical production may depend on whether it is the dominant or non-dominant language that is being inhibited (Blanco-Elorrieta et al. 2018). Thus, it is critical that future work investigate bimodal bilingual language behaviors in groups with varying language experiences and proficiency. Further, recent work on unimodal bilinguals has suggested that freely mixing languages can, in fact, be less effortful than staying in a single language (Zhu et al. 2022). Future research should investigate whether bimodal bilingual code-blending can be considered a form of free language mixing, absent any articulatory constraints, or whether the behaviors of bimodal and unimodal bilinguals are the result of qualitatively different systems for language regulation.

3.2.4. Language Regulation and L2 on L1 Effects

The traditional assumption has been that in sequential L2 learning, and especially L2 learning in adulthood after the L1 is well-established, the L1 has a privileged status. As discussed above, this assumption influenced the understanding of cognitive control in bilinguals and, for a long time, also meant that language regulation for adult L2 learners was understood to consist mostly or solely of regulating the L1, and especially limiting transfer from L1 to L2. More recently, studies have shown a preponderance of bidirectional language influences, and specifically that learning a second language has effects on the first language as well. This suggests that bidirectional language effects start emerging from the very beginning of second language learning, especially at the lexical level (see reviews in Kroll et al. 2021, 2022; see also Bice and Kroll 2015; Bogulski et al. 2019; Brice et al. 2021). What such findings show is that bilingualism, even at the earliest stages, changes the language system away from the monolingual system. In the new bilingual system, language regulation becomes a crucial component of successful communication, and it is possible that language regulation is central not only to regulating what has been acquired in the L2 but to the process of learning an L2 in the first place. Acquiring L2 proficiency may thus depend in part on how open the existing native language system is to influences from the L2, possibly recapitulating the openness of the language system in infants exposed to more than one language (Ferjan Ramírez et al. 2017; Petitto et al. 2012).

Given that proficient bimodal bilinguals experience cross-language activation similar to that of unimodal bilinguals, we might expect L2-on-L1 effects to be relatively independent of language modality, and, in fact, studies have observed L2 on L1 effects in L2 sign language learners. Casey et al. (2012) conducted a large-scale survey of novice ASL students and showed that over 60% of the learners self-reported sometimes using ASL signs when speaking. In the same study, the results of an experiment investigating narrative retellings, novice English L1-ASL L2 learners showed effects of learning to sign when they use English. Specifically, in face-to-face communication, they use co-speech gestures more (see also Weisberg et al. 2020; Emmorey et al. 2008a) than developing unimodal bilinguals, and as many as 25% of the learners produced one or more ASL signs alongside speech when retelling a story in English. This suggests that bimodal bilinguals experience lexical intrusions from ASL despite speaking to a non-signer and that this likely happens from the earliest stages of acquisition. Frederiksen (2021) similarly found that intermediate ASL learners accompanied English verbs of object placement (put, place) with co-speech gestures that were different from those used by non-signers. Specifically, the signers used handshapes that reflected the shape and orientation properties of the object being placed, and they acted so in a manner that is consistent with ASL but not with English. These findings suggest that not only the life-long exposure to and use of a sign language but also short-term exposure can affect the L1 by increasing and qualitatively changing gesture use as well as resulting in simultaneous use of the two languages. While it remains to be seen how patterns such as these evolve as proficiency increases, there is evidence that the functional brain networks supporting spoken L1 production look different in proficient L2 signers compared to monolinguals, suggesting that learning a signed L2 can have profound effects on language processing in the brain (Zou et al. 2012a) Nevertheless it is not yet clear how these patterns might inform the understanding of how efficient language regulation is acquired and how the ability to tolerate changes to the L1 affect the language learning process across modalities.

The relationship between L2 learning and L1 processing can perhaps be seen most clearly in studies investigating what happens to L1 access after using the L2 for shorter or longer periods of time. The asymmetrical switching costs discussed above showed that the L1 could be inhibited immediately after L2 use (e.g., Meuter and Allport 1999). Misra et al. (2012) found evidence that translation equivalents were inhibited in L1 after L2 use and that this inhibition lasted for longer than just a few trials (it lasted across at least two blocks, see also Van Assche et al. 2013). On a scale of months, a study by Linck et al. (2009) found that learners immersed in an L2 environment had temporarily reduced access to L1 while immersed. Moreover, the immersed learners showed an unexpected insensitivity to L1 lexical interference in a translation recognition task. This effect was found both during immersion and after returning to the L1 environment. The authors argue that these findings support an important role in the inhibition of the L1 in successful L2 acquisition. Other studies have reported similar findings (e.g., Baus et al. 2013; Brice et al. 2021; and see Kroll et al. 2018, for a review of the findings on language immersion). To date, inhibition effects have been investigated in bimodal bilinguals only on the shortest timescale, that is, in the context of cued language switching, where studies have found asymmetrical switch costs, suggesting that the spoken language is more strongly inhibited than the signed language. To date, no studies have examined the long-term effects or the effects of immersion on language access in bimodal bilinguals. As such, the answers are not clear to questions such as whether immersion in ASL reduces bimodal bilinguals’ access to English or whether their ability to use both languages simultaneously creates an opportunity for maintaining the relative activation levels of the two languages, regardless of language environment.

4. Language Environment and the Relationship between Language Regulation, Cognitive Control, and Language Processing

In the past decade, research has begun to explore the effects on the cognition of different bilingual experiences. Evidence is emerging to suggest that fine-grained variation in the contexts of language use affects the relationship between different linguistic and cognitive processes. Different language environments, both L1 and L2, pose different demands on language users, and recent years have seen a variety of metrics proposed to capture variation in bilingual’s language use and interactional contexts. Green and Abutalebi (2013) distinguish between bilinguals based on whether they use their languages mostly in separated situations (single-language contexts), consistently with different interlocutors (dual-language contexts), or whether they mix languages with the same interlocutor (dense code-switching contexts). Similarly, Beatty-Martínez and Titone (2021) propose different bilingual phenotypes based on patterns of language use and regulation in competitive vs. cooperative contexts. The entropy measure developed by Gullifer et al. (2018) considers the diversity of individuals’ social and language networks, with lower language entropy scores reflecting less diversity and thus more predictable contexts for the use of each language, and higher language entropy scores indicating the use multiple languages in a variety of contexts. The growing understanding of the importance of accounting for bilinguals’ contexts of language use in shaping the relationship between language and cognition provides a context for a better understanding of the theoretical importance of the existing evidence from studies of bimodal bilinguals and the questions we still need to ask. At the same time, this understanding highlights the unique contribution that studies of bimodal bilingualism can make to the field of bilingualism at large.

Studies have shown that various types of contexts affect bilingual language regulation, which in turn shapes control processes and their interaction with language processing (e.g., Pot et al. 2019; Gullifer and Titone 2020; Ooi et al. 2018). Gullifer et al. (2018) conducted a neuroimaging study of highly proficient French–English bilinguals with similar results. This study found that the bilinguals with lower entropy scores, indicating more predictability in when to use French vs. English, relied more on reactive cognitive control compared to bilinguals with high entropy scores, and low entropy bilinguals also showed less connectivity between different brain areas that have been found in previous research to be implicated in monitoring and language switching. Beatty-Martínez et al. (2020) investigated the relationship between lexical access and cognitive control in bilinguals who were alike in language proficiency but differed in their interactional contexts and the patterns of language regulation necessitated by these contexts. This study found that among Spanish–English bilinguals in compartmentalized language contexts where they generally only use one language per context, greater reliance on reactive cognitive control in the AX-CPT task correlated with better picture naming accuracy in L1 and L2. Among Spanish–English bilinguals who are immersed in the L2 (English) in environments where only some community members use both languages, there was overall greater reliance on proactive cognitive control processes and a positive correlation between more proactive control and higher picture naming accuracy in Spanish. For bilinguals in integrated language contexts, who can freely mix their two languages (code-switch) because they are in communities with speakers who are proficient in both languages and where code-switching is culturally acceptable, the pattern was in between those of the two other groups (Beatty-Martínez et al. 2020).7

The results of such studies suggest that it is long-term experience with a specific type of language regulation that shapes cognitive control abilities and their relationship to language processing. However, as is clear from studies of language immersion, bilinguals’ regulation needs can change when the language environment changes. The studies discussed above show that L1 access can change as a result of the type of language regulation required in contexts of immersion, and an important question is how such experiences affect cognitive control. Zhang et al. (2015) conducted a study on the effect of language switching training and showed that Chinese–English bilinguals living in China improved their performance on proactive control in the AX-CPT task after several days of being trained in switching between English and Chinese. Using the same paradigm with Chinese–English bilinguals in the U.S., however, revealed no proactive cognitive control improvements (Zhang et al. 2021), but crucially, the English-immersed bilinguals had much higher initial proactive control scores, suggesting that the experience of immersion and the resulting increased demands on language regulation had already led to the cognitive control benefits that the bilinguals in China obtained after language switching training. While more research is needed, the results of this study suggest that bilinguals who have a high need to inhibit either language in unpredictable contexts may be expected to most consistently show increased cognitive control abilities. Conversely, bilinguals with lower inhibitory needs or in highly predictable environments may be expected to experience fewer changes to their cognitive control abilities. From this perspective, bilinguals with experiences that look drastically different on the surface may, in fact, show similar adaptations. For example, bilinguals who do not need to fully inhibit one language, such as bimodal bilinguals, may look like those who can switch languages relatively freely, such as bilinguals in integrated or dense code-switching contexts.

From the perspective of understanding control and regulation abilities as a function of the demands of the language environment, the reason why bimodal bilinguals do not show cognitive advantages may thus not necessarily be due to the lack of perceptual or production overlap between their languages. Instead, it may result in part from bimodal bilinguals rarely needing to decide which language to use in any context. In an English-speaking context, a bimodal bilingual might supply signs for lexical items while speaking without disrupting communication because such signs are likely to be perceived as co-speech gestures by a non-signing interlocutor. In fact, Pyers and Emmorey (2008) showed that CODAs do not fully suppress ASL even when their productions (i.e., grammatical facial expressions such as furrowed eyebrows) could be construed by the interlocutor as conveying negative emotions or attitudes in the context of speaking in English.

As discussed previously, the experience of learning a signed second language may lead to changes in cognitive abilities in some cases (Macnamara and Conway 2014). It is also clear that cognitive control plays a similar role for bimodal and unimodal bilinguals during language processing. Giezen et al. (2015) investigated whether bimodal bilinguals showed a correlation between non-linguistic conflict resolution (measured by the Stroop effect) and efficiency of managing cross-language activation, similar to unimodal bilinguals (Blumenfeld and Marian 2013). Giezen and colleagues found that bimodal bilinguals who had smaller Stroop effects (showing better inhibitory control) also experienced less competition from ASL in recognizing English words. This result suggests that although bimodal bilinguals look like monolinguals in their cognitive control abilities, they may use their abilities as unimodal bilinguals do. This raises the possibility that the regulation and control patterns of some bimodal bilinguals may look more like unimodal bilinguals than others. As the studies of unimodal bilinguals by Zhang and colleagues showed, bilinguals’ language environment affects their cognitive control abilities, but what this relationship looks like for bimodal bilinguals is as of yet unknown. The majority of the evidence from bimodal bilinguals comes from CODAs and proficient L2 learners in mostly English-speaking contexts, but experiences may vary between bimodal bilinguals. Some bimodal bilinguals may be switching regularly between ASL and English contexts. Others may be using ASL in the Deaf community, where it may not be culturally appropriate to voice in English. Such contexts may involve regularly switching languages and an increased need to inhibit the non-selected language and may consequently result in different cognitive control adaptations. It is also important to acknowledge that many highly proficient bimodal bilinguals are professional interpreters. Studies of this group of language users, therefore, tend to include interpreters whose experiences with language activation and inhibition are likely to differ from non-interpreters (but see Christoffels et al. 2006 for evidence that interpreting experience in unimodal bilinguals does not alter primary language processes above and beyond their high proficiency). Whether the inclusion or exclusion of professional interpreters, as well as considerations of situational changes in the individual, such as whether an interpreter is at work or not, would affect the basic cognitive and linguistic patterns identified in previous studies is an open question. Another open question concerns whether there are differences in terms of the presence and strength of language co-activation between CODAs vs. bimodal bilinguals who learned the sign language in a classroom as adults vs. those who learned in contexts of language immersion.

Dynamic Recruitment of Cognitive Control during Language Processing

Recent studies have investigated dynamicity in the relationship between language processing and cognitive control. By using eye-tracking, Hsu and Novick (2016) investigated how efficiently monolingual English speakers were able to revise incorrectly parsed sentences as a function of their level of cognitive control engagement. High vs. low engagement contexts were created by interleaving congruent and incongruent Stroop and language processing tasks. When the sentence processing task took place during high cognitive control engagement (following an incongruent Stroop trial), the initial incorrect parse was corrected faster than in low cognitive control engagement contexts. This result suggests that monolinguals’ language processing benefits from dynamically engaging cognitive control (see also Hsu et al. 2021; Thothathiri et al. 2018). However, similar studies with bilinguals have not revealed the same benefit of dynamic cognitive control engagement for language processing. Specifically, bilinguals appear to take advantage of their experience with language regulation to activate a high degree of cognitive control, irrespective of the experimental manipulation. Crucially, the effect of cognitive control recruitment in bilinguals appears to suffice to overcome any potential disadvantages associated with language co-activation, such as the well-documented delays in lexical processing observed in bilinguals compared to monolinguals (see Bialystok 2009). What has not yet been discovered, however, is how a bilingual’s language environment and their experience with language regulation may affect their recruitment of cognitive control for the purposes of language processing. Bimodal bilingualism does not confer the boost to cognitive control abilities that unimodal bilingualism does, and we might therefore expect to see bimodal bilinguals’ processing benefit from the dynamic engagement of cognitive control. At the same time, work by Giezen and Emmorey (2016) provides evidence that the bimodal bilinguals’ ability to use both their languages at the same time does not appear to mitigate the typically observed disadvantage in lexical access in bilinguals compared to monolinguals (Bialystok et al. 2008; Gollan et al. 2002; but see Blanco-Elorrieta and Caramazza (2021) for an account that assumes that language processing requires the same level of effort for monolinguals and bilinguals). This could disadvantage bimodal bilinguals’ language processing compared to monolinguals’, while at the same time, bimodal bilinguals are not expected to show the same ability to recruit cognitive control abilities to support language processing as unimodal bilinguals are. Such issues are among the many unanswered questions about bimodal bilingualism, which offer the potential to better understand the interactions between language experiences, language regulation, cognitive control, and language processing.

5. Conclusions

The preceding sections have presented an overview of what the study of bimodal bilingualism contributes to our understanding of bilingualism more generally and in particular to understanding the interactions between language regulation, cognitive control, and language processing.

Bimodal bilingualism can be a tool for investigating questions about language and cognition that are less approachable when studying monolinguals or unimodal bilinguals. There is by now a substantial body of research that has investigated language behaviors, language switching, and cognitive control in proficient bimodal bilinguals. Among other things, this research has revealed which aspects of bilingualism are subject to modality effects and how the language system handles two languages when they can be used at the same time. Crucially, this work has also provided the means to establish clearer links between bilingual experiences and adaptive outcomes. Yet, for all that the research on bimodal bilingualism has contributed in the last decades, many questions still remain unanswered.

In this paper, we have highlighted the value of investigating bilingual variation. It will be an important endeavor for future research on bimodal bilingualism to grapple with the questions of variation and how the language environment affects language use, regulation, and cognitive adaptations in individuals who sign and speak. Another avenue of research that should be pursued relates to sign language learning. While most of the current research has focused on proficient bimodal bilinguals, there are important implications to understanding learning in bimodal bilinguals. Does knowing a signed second language make subsequent spoken language learning easier? Are unimodal bilinguals advantaged in learning a sign language? To what extent does the ability to regulate two spoken languages affect the nature of language regulation in an additional signed language? We noted, for example, that language immersion in a spoken L2 results in long-lasting inhibition of the L1. An important question for understanding the role of L1 regulation for L2 learning is whether immersion in a sign language produces similar effects and whether outcomes are similar in learners with and without prior spoken L2 proficiency. Questions such as these hold great promise for revealing the interactions between bilingualism, cognition, and language learning ability.

In sum, bilingualism provides a lens through which to investigate plasticity in the interactions between language and cognition that may be obscured in monolinguals. Research from the last decade has shown that variation in bilingual language experiences impacts minds and brains, and it has become clear that an approach that embraces variation in language proficiency, language environment, different types of contexts, as well as in language modality is necessary to reveal the plasticity of which the human brain is capable.