1. Introduction
Working memory (WM) refers to the human ability to maintain information in an active state while performing cognitive tasks with that information. The most widespread theory of WM, multicomponent
WM, holds that WM consists of multiple interrelated processing systems controlled by a central executive, which decides the importance and therefore the availability of the representations stored within these processing systems (
Baddeley & Hitch, 1974; see
Cowan, 2017 for a review of WM). The processing systems, which vary across theoretical frameworks but generally include a phonological buffer, a visuospatial buffer, and an episodic buffer, are distinguished by the computational processes by which they store and perform operations upon representations. Therefore, a task’s demands determine which processing systems are the most effective.
The phonological buffer is also called phonological short-term memory (STM). Whereas WM refers to the entire process of the encoding, storage, and retrieval of relevant information in many different forms, phonological STM refers specifically to the storage of serially ordered information as a phonological code in the phonological buffer (
Hall & Bavelier, 2010). The phonological and serial nature of STM means it is uniquely optimized for the representation and manipulation of spoken linguistic units. The phonological and grammatical content of spoken language is, by necessity of the means of production and comprehension in the auditory–verbal modality, arranged sequentially in temporal chunks of information. Temporal order information is key for parsing spoken language, as well as other nonverbal cognitive skills, such as cause/effect understanding (
Hall & Bavelier, 2010). STM interacts with the articulatory loop, which is a WM process that enables speakers to maintain verbal information in their memory by covertly or overtly reciting the information (
Hall & Bavelier, 2010).
1.1. Working Memory in DHH Individuals
Many studies have investigated how executive functions operate in deaf and hard-of-hearing (DHH) children and young adults; they have often found that DHH individuals are at risk of diminished executive function relative to their hearing peers (e.g.,
Jamsek et al., 2022;
Kronenberger et al., 2014;
Marschark et al., 2016;
McCreery & Walker, 2022). However, studies of signing DHH children have typically found that early access to sign language input appears to be a protective factor (e.g.,
Hall et al., 2018;
Goodwin et al., 2022;
Delcenserie et al., 2024). Studies which specifically investigate STM often aim to determine whether STM differences explain, or are explained by, the difficulties that DHH individuals may experience in acquiring spoken language. These studies commonly use serial recall tasks, in which participants are asked to recall spoken or visually presented lists of items (such as digits) in the order in which they were presented.
Cleary et al. (
2001) compared 45 DHH cochlear implant (CI) users with 45 typically hearing controls, all aged 8–9 years, as they completed a serial recall task with patterned shapes, auditory labels, or both. They found that the DHH group achieved shorter span scores overall and did not benefit from the addition of auditory labels as much as the typically hearing group did. The shorter forward digit spans in such DHH children has also been positively correlated with spoken and written language outcomes (
Pisoni & Cleary, 2003;
Pisoni & Geers, 2000). While the analyses in those papers did not determine causal effects, the authors concluded that such differences in verbal WM capacities could explain the interindividual variability in speech and language outcomes among this population of CI users.
Pisoni and Cleary (
2003) also compared children in oral-only and total-communication environments and found that the oral-only children had longer forward digit spans, which they interpreted as evidence that spoken language exposure supports the development of phonological STM.
DHH people who primarily use a natural signed language to communicate often display a disadvantage on serial recall tasks requiring recall of
ordered sequences, but not free recall tasks, as compared to their hearing peers (
Bavelier et al., 2008). DHH signers exhibit a sign-based encoding and rehearsal of items into their STM, as evidenced by the phonological errors they make in serial recall tasks, which cohere with the phonological parameters of their signed language (
Bellugi et al., 1974). Signed languages exhibit coarser temporal information and longer articulation times as compared to spoken languages, which have been cited as the reasons for serial recall differences between signers and speakers.
Marshall et al. (
2015) studied WM abilities in DHH children with DHH parents (henceforth, DoD children), DHH children with hearing parents (henceforth, DoH children), and typically hearing children aged 6–11 years. They administered two WM tasks: the Spatial Span Task, which requires the participant to tap a sequence of blocks in the same or reverse order as shown by the examiner, and the Odd-One-Out task, which requires the participant to recall the location of one shape which differed from two other shapes shown in a grid. They found that on both tasks, the typically hearing and DoD children performed comparably, whereas the typically hearing children outperformed the DoH children. They also found that when controlling for age and nonverbal reasoning, the children’s expressive vocabulary (measured with the Expressive One Word Picture Vocabulary Test) predicted their WM task performance. They concluded that early and robust language experience, rather than deafness, impacts the development of nonverbal executive-loaded WM.
1.2. The N-Back Task: A Measure of WM
The n-back task is a test of the STM for serial sequences, which demands more executive processing than typical serial span tasks. In an n-back task, the participant views a continuous stream of items belonging to a delimited group (such as digits, letters, or images) and responds whenever the current item is identical to what appeared n items before. For a 1-back task, it is the immediately prior item (meaning two identical items appeared consecutively); for a 2-back task, the items are separated by one item; and so on, with greater values of n requiring greater executive and storage capacity.
Unlike serial span tasks, the n-back task is a continuous performance measure during which the participant cannot know which stimuli are relevant until after they disappear. Therefore, decisions about the relevance of the stimuli must be made while they are held in the WM. The dynamic format of the n-back task could be expected to attenuate reliance on verbal chunking and the rote rehearsal of information, which are the heuristic strategies theorized by
Hall and Bavelier (
2010) to give speakers an advantage over signers on serial span tasks requiring the ordered recall of fixed spans. Therefore, the n-back task can be considered a measure of the serial STM capacity, which minimizes the use of heuristic shortcuts commonplace to traditional serial span tasks.
The n-back task requires the participant to both maintain the serial order information in their STM and dynamically recall the items in serial order according to the task-based goals. For the 2-back test, participants must suppress irrelevant information (the immediately prior item) while maintaining it in their memory as the subsequent 2-back comparison item. Thus, the
n-back task, particularly at higher ns, incorporates what
Cowan (
2017) calls attention-control WM, which refers to using attention to preserve the task-relevant information and inhibit the irrelevant information. These attention-control mechanisms are domain-general, and distinct from passive buffers, such as STM, which are domain- and modality-specific (
Engle et al., 1999).
There have not been any behavioral studies that have studied WM in DHH people using visual
n-back tasks. However, the n-back task is frequently used as a WM task in neuroimaging studies due to its simplicity and easy parameterization of difficulty (by increasing the
n) (
Pelegrina et al., 2015).
Cardin et al. (
2018) scanned DHH adults with a functional MRI as they performed 2-back tasks with point-light display stimuli, which were either linguistic (British Sign Language [BSL] signs) or nonlinguistic (moving objects). They aimed to determine whether the observed differences in neural recruitment during the WM tasks were due to the linguistic nature of the stimuli, the variation in the language experience of DHH people, or truly due to the absence of auditory experience. They found that the superior temporal cortex was recruited for the 2-back task but not the control task in only the DHH group, regardless of the linguistic nature of the stimuli. They hypothesized that the superior temporal cortex, which is an auditory processing area in typically hearing people, underwent a cross-modal reorganization to then subserve domain-general WM in DHH people, allowing for a reduced cortical recruitment of the parietal and frontal regions, which are typically recruited for WM in typically hearing people. These neural changes in the DHH group were accompanied by faster reaction times (but no change in accuracy) for the DHH group as compared to the hearing groups for only the 2-back task, but not for the control task (detecting whether any dots in the display were yellow). However, it is important to note that each item in the 2-back task (either a BSL sign or an object) was a dynamic point-light display, which elicits spatial motion processing. Accordingly,
Cardin et al. (
2018) interpreted the reaction time difference as more in line with previous studies showing a DHH advantage on tasks involving dynamic visuospatial information, rather than serial recall per se (
Keehner & Atkinson, 2006).
1.3. Current Study
This study is the first to systematically examine the performance of DHH children on the n-back task. We examined the longitudinal changes in performance on visual 1-back and 2-back tasks among a sample of 103 DHH signing children aged 7–11 years across a period of two years. The use of an accelerated longitudinal design allowed us to cover an age range of 7 to 13 years. In this first longitudinal study of WM development in DHH children, we examined the developmental changes in their n-back WM performance and determined whether their performance on these tasks was predicted by their family background and language ability. We predicted that their task performance would improve across the age range tested, with larger age-related gains for the children from deaf families compared to those from hearing families (reflecting the positive effect of strong early language exposure). By including a standardized measure of sign language ability—the ASL Receptive Skills test—we further sought to determine whether any effect of the parental hearing status was mediated by language ability.
2. Materials and Methods
2.1. Participants
The participants (N = 103) were students who studied at one of five bilingual–bicultural schools for the deaf across the United States. All the schools employed American Sign Language (ASL) as the primary language of instruction and interaction, and English as the primary written language. The participants were recruited through the schools’ communications with parents, including social media posts, letters, emails, and at school events or after-school pickups. The study was approved by the Institutional Review Board of Rochester Institute of Technology and by the research review boards of all the participating schools.
The participants were between 7 and 11 years of age upon inclusion into the study. To be included, they had to have some degree of diagnosed hearing loss as well as their primary language of instruction being ASL. The participants were excluded if they had documented learning or intellectual disabilities or mobility issues that would preclude them from performing the tasks. A reported ADHD diagnosis was not grounds for exclusion; 10 participants had a reported ADHD diagnosis. Those 10 children did not differ from the rest of the sample in their demographics or task performance.
Being enrolled in bilingual schools for the deaf, the children included in this study shared a commonality of having sustained language input in at least one accessible language from an early age up until when the study was conducted. The children were assessed with the following language and academic measures (detailed descriptions under the
Measures section): ASL comprehension (ASL-RST), a measure of spoken English ability (OWLS-LC), self-reported hearing level, and nonverbal IQ (KBIT-2). The demographic data for all the children are reported in
Table 1; the language assessments, nonverbal IQ, and audiological data are presented in
Table 2.
2.2. Measures
2.2.1. N-Back Working Memory Task
The
n-back experiment used in this study was developed in-house with the Paradigm software program for building and running psychological experiments (
Perception Research Systems, 2007). The experiment constituted two tasks that were wholly identical besides the
n, which dictated which items were the targets: first the participants performed a 1-back task, then a 2-back task. In the 1-back task, an item was a target if it was identical to the item immediately prior; in the 2-back task, an item was a target if it was identical to the item appearing before the prior item. The items were cartoon images of “farm animals” (with 11 different types of animals), which appeared in one location on the screen, set against a cartoon “farm” background (see
Figure 1).
A repeatable practice block preceded each experimental block, with the 2-back practice block occurring after the 1-back experimental block. The experimenter encouraged the child to repeat the practice block if they saw that the child had not yet understood the instructions; this was common for the 2-back task. Each experimental block contained a randomized order of items, which changed upon each iteration of the block. The blocks ended after 10 targets appeared (regardless of whether the child correctly responded to them or not) or after 50 total trials, whichever occurred first. Nine percent of the 1-back tasks and eight percent of the 2-back tasks reached the maximum of 50 total trials without 10 targets appearing. Each item appeared for 1000 ms, with inter-item intervals of 1000 ms.
2.2.2. OWLS-II Listening Comprehension Subscale
Their spoken English receptive skills were assessed using the OWLS-II Listening Comprehension Subscale (
Carrow-Woolfolk, 1995). For the OWLS-II LC, children listen to a spoken sentence and select the best-matching picture from four possible options. The items assess lexical/semantic, syntactic, pragmatic, and supralinguistic comprehension. Since the researcher was deaf, the items were presented as individual video files on a laptop, for which a hearing colleague spoke each item. The laptop was positioned in front of each child with a stereo speaker placed in front of the laptop; the response book was next to the laptop. Each child was prompted to watch each video without touching the table or speaker to feel the vibrations (a common request). The children wore their hearing aids during the test only if they reported regular hearing aids use. After completion of the test, the children were reassured that their performance on the test was not critical (to minimize the stereotype threat or associated negative feelings). The raw scores are reported in lieu of standardized scores because no standardized population norms exist for the test and the minimum standardized score of 40 obscures much of the variability in the raw scores.
2.2.3. ASL Receptive Skills Test
ASL receptive skills were assessed using the ASL Receptive Skills Test (ASL-RST;
Enns et al., 2013). The ASL-RST comprises 42 items, for each of which an ASL sentence is given and the child selects the best-matching illustration from four possible options. The ASL-RST was designed for use with children aged 3–12 years.
2.2.4. Kaufman Brief Intelligence Test, Second Edition (Matrices)
Nonverbal intelligence (NVIQ) was assessed using the Matrices subtest of the Kaufman Brief Intelligence Test, Second Edition (KBIT-2), which is the only nonverbal subtest within the KBIT-2 (
Kaufman & Kaufman, 2004). Each item contains a visual puzzle with an incomplete segment, and six possible answers. Only the correct answer will perfectly complete the pattern presented in the puzzles, and the items increase in difficulty, either by expanding the matrix and/or by presenting more complicated patterns. All the KBIT-2 scores were converted to standardized scores for the analyses in this study.
2.3. Procedure
All the testing was performed at the schools, either during or after the school day, as determined by the schools’ testing or administrative departments and/or parental preference. On rare occasions, testing was performed in the home upon parental request. The researcher visited each school a total of four times, with each visit lasting 2–4 weeks. They met with each child participating in the study for 1–2 testing sessions per visit (described below). The researcher who administered the tests was deaf and ASL-fluent. The testing took place while the researcher and child were alone in a distraction-free room. The instructions were administered in ASL.
Each child received a battery of attention, language, and cognition tests that comprised a larger project; the n-back task reported here was one of four cognitive tests given during each of the four visits. The tests were all given during a one-hour session upon every visit, for a maximum of four cognitive assessment sessions per child. Due to the logistical limitations of travel and testing, the four visits for data collection were not spaced out uniformly (time between visits 1–2: M = 248 days, σ = 14.8; visits 2–3: M = 172 days, σ = 7.33; visits 3–4: M = 236 days, σ = 40.2). Because of this, as well as the staggered ages of the participants included in the study, all the longitudinal models were calculated using the centered age as the time parameter rather than the number of the visit.
All the visits contained at least one session, during which cognitive tests were administered. For each child, a second session was administered during the visit that was the closest possible to the child’s 10th birthday. Language and NVIQ measures were obtained during this second session, which was administered at least 24 h after a cognitive assessment and no more than 2 weeks following a cognitive assessment. The order of the assessments was counterbalanced in all the sessions, except for the language measures. The ASL and spoken English assessments were always given at the end of the session to avoid a potential stereotype threat from administering tests of spoken and signed language to DHH children in a bilingual–bicultural environment.
2.4. Apparatus
The n-back task was administered using Paradigm experimental software, running on a Getac F110 tablet computer with an integrated 11.6″ HD touchscreen (resolution: 1366-by-768 pixels; brightness: 800 nits) and a Windows 7 operating system. The responses were recorded via touchscreen. The language tests were administered using a Dell Latitude E7470 14″ laptop; for the audio during the spoken language tests, the laptop was connected to a Bose SoundLink Mini II stereo speaker placed on the table in front of the participant.
2.5. Statistical Analyses
All the statistical analyses were conducted using R version 4.4.3 running within Posit R Studio version 2024.12.1+563. Linear mixed models were performed using the lme4 toolbox (version 1.1-35.5;
Bates et al., 2015). The full analysis script is available on the Open Science Framework—see the Data Availability Statement.
4. Discussion
In this paper, we sought to assess the performance of DHH children on a task that measures working memory (WM) while minimizing the ability of the children to use heuristic shortcuts, which may lend an advantage to the children who use a spoken language. While such an advantage is most pertinent when comparing deaf versus hearing children, it may also confound the results within an entirely DHH group (such as in this study) if some individuals within this group have stronger spoken language skills than others. We chose the n-back task, which requires children to recall items presented prior to the current item. Since the children could not know which items were relevant (i.e., the first item of a matching pair) until after the appearance of the next or second-to-next paired item, we argue that this task limits the use of heuristic shortcuts to boost task performance. To assess the development of WM skills, we collected data from each child across multiple time points. We also examined the influence of early childhood language environment and skill by including the family background and the child’s performance on an ASL receptive skills assessment in our models.
We found that, as expected, the 2-back task was more challenging than the 1-back task, but that this was not influenced by age. The children in our sample individually demonstrated an improvement in performance across the ages of 7–13 years, indicating that our WM task captured the development in WM capacity for this age range. When assessing the influence of language on WM capacity, we found that the DHH children with DHH parents demonstrated accelerated improvement in WM capacity. We also found that this relationship was mediated by ASL skill.
We present two main findings in this paper. First, we recommend the
n-back task as a possible alternative to tasks, such as serial span tasks, when assessing WM capacity in children who do not use a spoken language as their first language. Serial span tasks allow for articulatory-based heuristic strategies, which may bias such measures against signing children. This was suggested by
Hall and Bavelier (
2010), who proposed the
multiple coding hypothesis: speakers depend on the articulatory loop much more than signers for encoding information into WM, whereas signers rely on more distributed coding across signed phonological, visuospatial, or episodic processes. They suggested that instead of truly reflecting serial WM capacities, serial span tasks reward a reliance on verbal chunking and the rote rehearsal of information, which are articulatory heuristic strategies that give hearing speakers an advantage over DHH signers on the ordered recall of fixed spans. However, as we did not explicitly compare the
n-back task with serial span tasks, we recommend that future research compare the performance of DHH and hearing children on each type of task to determine whether disparities in serial span performance are replicated on
n-back tasks.
Second, we suggest that our data support the importance of a strong foundation of accessible language instruction for the development of cognitive skills, such as working memory. Specifically, we emphasize that signed language instruction supports the development of cognitive skills often thought to be specifically scaffolded by
spoken input, such as serial STM. One difficulty inherent in comparing DHH children with DHH parents to DHH children with typically hearing parents is that these two populations of children vary in multiple ways. A key difference is in the timing and quality of early accessible language input. Children with DHH signing parents are typically exposed to phonologically and syntactically rich ASL from birth, whereas hearing parents who choose to sign with their DHH infants will (i) initiate that language input after the detection of hearing loss and (ii) will usually be learning the language at the same time as they are using it to communicate with their deaf infant and have beginner-to-intermediate language learner ability (
Pontecorvo et al., 2024). The significance of this latter difference has recently been called into question by research demonstrating similar rates of ASL vocabulary acquisition in DHH children regardless of the hearing status of their parents if exposure starts prior to 6 months of age (
Caselli et al., 2021). However, for some DHH children with typically hearing parents, early communicative interactions may be disrupted, resulting in executive function deficits (
Morgan & Dye, 2020), increased parenting stress (
Quittner et al., 2010), and differences in parent–child interactions (see
Curtin et al., 2021 for a review). In the current study, controlling for ASL receptive skills removed the DoD versus DoH parent group differences in the growth of working memory performance. However, assuming that ASL receptive skills are correlated with these factors due to the influence of early communicative interactions, we should be cautious in focusing any interventions solely on language instruction. The timing and quality of language input, as well as the resulting parent–child interactions driven by that input, are likely to play crucial roles in scaffolding cognitive development in DHH infants and young children.