Cross-Domain Input, Mutual Exclusivity, and Inferential Reasoning: When LLMs Learn Words Like Humans

Abstract

Humans acquire meaningful language by storing perceptual categories, category-word mappings, and conditional IF–THEN rules in rich, cross-domain, multimodal contexts. Crucially, structured cross-domain input that pairs visual context and language (text) appears to be fundamental to this process, enabling individuals to acquire, for example, the lexicon. The present study investigates whether Large Language Models (LLMs) can learn words when trained on structured cross-domain input rather than text-only exposure. This paper incorporates a controlled Fictitious-Animal Paradigm featuring 32 creatures and 32 pseudowords, divided into two phases: single-animal and dual-animal scenes. Single-animal contexts entailed storing categories and mappings, while dual-animal environments involved storing logical constraints and performing inference. The authors propose a computational model centred on four capacities for context-based word learning: storing perceptual categories, category-word mappings, and the Mutual Exclusivity rule (formalised as a conditional IF-THEN statement), and retrieving stored information to assign novel labels through inference. The evaluation demonstrates that structured cross-domain input enables LLMs to exhibit behaviour consistent with conditional IF–THEN rules, allowing them to infer and acquire novel words based on current visual contexts. These findings suggest that, with this input that integrates context and text, LLMs display adaptive, real-time human-like inferential reasoning in word learning.

1. Introduction

In natural settings, humans acquire meaningful language in rich, cross-domain, multimodal contexts. Rather than assuming full integration across sensory modalities such as vision and audition, the authors operationalise cross-domain input as the presentation of scenes containing fictitious animals alongside written sentences. Thus, in this study, structured cross-domain input refers to the pairing of complementary non-linguistic and linguistic information: visual context and language (text).

By intersecting insights from disciplines such as neuroscience, cognitive and developmental psychology, and linguistics, the authors conceptualise an architecture that flows from word learning to the acquisition of linguistic motor programmes and comprises four deeply intertwined developmental milestones in human language acquisition. This theoretical framework is mainly rooted in the learning of motor programmes—integrative cerebral action schemas that, as posited by Lashley [1], Fuster [2], and Graybiel [3], stipulate the words or acts to be assembled into a particular sequence, their planning (syntax), and their final production (output). First, humans detect incoming visual stimuli and group shared physical traits into stable perceptual categories. Second, they associate these categories with linguistic labels, exploiting the synchronous co-occurrence of visual scenes and linguistic stimuli. Third, the human brain registers and stores conditional IF–THEN constraints over these category-word mappings to resolve referential ambiguity—a logical process best exemplified by the Mutual Exclusivity rule discussed later in this section. This third milestone extends to storing higher-order conditional IF–THEN statements, including syntactic patterns and grammatical rules. Fourth, the brain retrieves previously stored information, including conditional IF–THEN rules, to deploy inferential reasoning for word learning and to plan sentences via motor programmes. The present study applies this four-milestone architecture to word learning in Large Language Models, leaving the acquisition of syntactic patterns and grammatical rules for future research.

The authors distinguish between referents, categories, and labels. Referents denote the individual objects presented in the visual scene (i.e., specific instances of animals). These referents belong to perceptual categories, defined as sets of objects sharing visual properties that support generalisation across instances. Labels refer to the novel lexical items (words) introduced during the task, which must be mapped onto these referents.

Methodologically, the present paper introduces a Fictitious-Animal Paradigm as an experimental design that features 32 creatures and 32 pseudowords for word learning in a Large Language Model. Each creature constitutes a single exemplar of a distinct category and, therefore, functions as an operational stepping stone toward full categorical abstraction. Multi-exemplar variations (such as colour and morphology) are relegated to future research. In turn, each pseudoword (e.g., flug or gint) serves as a category label.

Building on this foundation, the guiding function of visual context becomes paramount under conditions of referential ambiguity, where multiple distinct referents in the environment may correspond to a given expression or name. In these uncertain scenarios, human and artificial interpretation is achieved not by random mapping but through a systematic process of logical elimination, in which the current visual context serves as a constraint that prevents incorrect label assignments.

A key principle in word learning is Mutual Exclusivity, a constraint first proposed by Markman and Wachtel [4], and formalised here as a conditional IF–THEN rule: “IF an object or perceptual category (C₁)—is already associated with a word (W₁), THEN it cannot receive a new word (W₂).” The authors represent this rule as ‘IF C₁ == W₁ → THEN C₁ != W₂’, which essentially means ‘One word per referent or category’.

To evaluate the model’s ability to learn words when trained on structured cross-domain input, the Fictitious-Animal Paradigm features 32 creatures and 32 pseudowords, divided into two phases: single-animal scenes and dual-animal visual contexts. Single-animal scenes entailed storing categories and category-word mappings, while dual-animal scenes involved storing logical constraints, retrieving the stored information, and executing inferential reasoning. Specifically, the authors investigate the following four computational capacities (Ks) for word learning:

• Storing a Perceptual Category (K1): The model must detect the visual features of a single creature and cluster them into a stable referent, which serves as the representational basis for a category (C₁). Essentially, the model stores these visual features before linking them to any linguistic label.
• Storing a Category–Word Mapping (K2): The model must form stable associations or mappings between the perceptual features of the category and its corresponding linguistic input (label, e.g., “flug”). This process draws on the synchronous co-occurrence of visual scenes and linguistic inputs, functioning as the representational substrate of a category-word mapping (C₁ == W₁).
• Storing the Mutual Exclusivity Rule (K3): The model must register and store the constraint that each category must possess a unique label. The Mutual Exclusivity rule is formalised as a logical constraint: IF C₁ == W₁ → THEN C₁ != W₂ (if this animal is a “flug”, it cannot be a “gint”).
• Deploying Inferential Reasoning (K4): The model must retrieve the previously stored information (K1–K3) and execute the final resolution (K4): IF C₁ == W₁ → THEN C₁ != W₂ → Resolution: C₂ == W₂.

In this view, structured cross-domain input is paramount, as the model stores and retrieves non-linguistic and linguistic information to perform inferential reasoning and acquire words grounded in current visual environments. Initially, the artificial neural network stores three components: a perceptual category—represented here by a single exemplar—(K1), its corresponding category-word mapping (K2), and the Mutual Exclusivity rule (K3), formalised as a conditional IF-THEN statement. Rule learning occurs when an unfamiliar creature is presented along with a novel name. Subsequently, the stored information (K1–K3) is retrieved to exclude the familiar animal and assign the new label to the unfamiliar creature through inference (K4). Note, therefore, that K1 and K2 operate in the single-animal phase. However, K3 and K4 work in the dual-animal phase, since both storing the rule and deploying the inference require a context that allows for comparison between two entities. More precisely, in this dual setting, the model contrasts the familiar creature with the unfamiliar one, excluding the familiar creature (C₁ != W₂) and assigning the novel pseudoword to the new entity (C₂ == W₂). Figure 1 illustrates the single- and dual-animal scenes used in the current study.

Together, these capacities (K1–K4) enable the model to process structured cross-domain input (visual context and written text), exhibit behaviour consistent with conditional IF–THEN rules, execute inference via exclusion, and acquire novel words. Thus, the model displays adaptive, human-like inferential reasoning in word learning, adjusting its responses to real-time needs derived from current visual scenes.

Related studies investigate and apply learning mechanisms within artificial neural networks and Large Multimodality Models (LMMs) to analyse how these systems process, categorise, and map visual context and language. Specifically, Sun et al. [5] explored similar concepts to build databases oriented toward LMMs by enhancing descriptive captions with visual attributes for multimodal perception. Expanding beyond database construction, we propose a framework to train LMMs to acquire new computational capacities for word learning. Furthermore, while Sun et al. [6] addressed the importance of prompt selection for improving visual in-context learning, our research adapts prompts to help the model store and implement four distinct computational capacities. In turn, Tang et al. [7] utilised a visual description step to accurately describe objects within visual scenes in LMMs—as seen in structured visual reasoning for perception policy learning—an approach that serves as a viable solution for training them on specific visual tasks like robotic object manipulation. Regarding human word learning, Vong and Lake [8] demonstrated through cross-situational word learning with multimodal neural networks that artificial neural networks can learn word-referent mappings from a single epoch of training—mimicking the data efficiency of cross-situational word learning, but noted limitations in reasoning via Mutual Exclusivity. In contrast, the present study introduces a proposal for acquiring four computational capacities that enable inferential reasoning, including Mutual Exclusivity. Finally, Nortje et al. [9] demonstrated that visually grounded speech models exhibit a Mutual Exclusivity bias. They found that this bias can be acquired when enhanced prior knowledge is integrated into vision branch initialisation. We confirm this effect and consider it in the visual training of the BLIP-2 model. Unlike these studies, the present work introduces a new framework that combines four computational capacities (K1 to K4) in a new task configuration and provides a supervision strategy that overcomes the reasoning limitations observed in previous models.

The following sections review the theoretical foundations underlying this study. First, neuroscientific accounts of associative learning and conditional IF–THEN structure are examined, followed by cognitive perspectives on conditional reasoning. Subsequent sections discuss developmental evidence on referent–word mapping, Mutual Exclusivity, and rule abstraction, before turning to the role of visual context in supporting conditional inferential reasoning. These perspectives converge on a set of theoretical premises and the Fictitious-Animal Paradigm, which provides a controlled framework for examining iterative conditional inferential reasoning in artificial systems.

1.1. Neuroscientific Foundations: Neuronal Networks, Including Conditional IF–THEN Rules

The foundations presented here informed our neurobiologically inspired theoretical framework and the Fictitious-Animal Paradigm (see Section 2). For example, repetition drives learning through Hebbian principles and Long-Term Potentiation, a cellular process that encodes and stores memories thanks to repeated practice. Our specific focus on Large Language Models mirrors this architecture: ensuring the network can successfully encode and store cross-domain input, flawlessly binding visual context with written textual input to perform inferential reasoning and learn words.

The brain is extremely plastic. Neuroplasticity is the term used to describe the brain’s highly robust and well-developed capacity to change, forming and strengthening some neuronal connections and weakening or eliminating others [10]. Natural learning environments typically provide cross-domain, multimodal input, in which non-linguistic and linguistic information is coupled, combining signals across multiple sensory modalities (e.g., visual and auditory). For example, when a small boy sees a bird for the first time and his caregiver points to it while saying “That’s a bird”, some neurons in the child’s brain form a new connection. Each time the experience is repeated, neurons can strengthen their synapses [11,12,13]. This enables associative learning through the principle commonly summarised as “Neurons that fire together wire together” [14]. The more these neurons are refired, the more permanent the new memory becomes. This biological strengthening over time is driven by Long-Term Potentiation, the cellular process that transforms transient experiences into permanent memories. When a person repeatedly activates a group of neurons—for example, by practising a skill—these cells undergo profound functional and structural changes. At the functional level, the first (presynaptic) neuron releases more neurotransmitters when an electrical signal (action potential) reaches it. In contrast, the second (postsynaptic) cell inserts more receptors into its membrane, making synaptic transmission significantly faster and more efficient. This explains why complex tasks, like playing a musical instrument or driving a car, become effortless over time. At the structural level, if this paired activation is repeated enough, new dendritic spines sprout to link the neurons via entirely new synapses, physically consolidating the neural pathway [11,15,16,17,18]. Figure 2 illustrates the unidirectional propagation of an action potential from the presynaptic neuron to the postsynaptic neuron. It also depicts the transition from the initial synaptic structure to a reinforced state, characterised by increased neurotransmitter release that results from repeated activity.

Building on Hebb’s idea, Fuster [19] describes neuronal networks as units of memory or knowledge formed from stimuli that are “experienced at, or nearly at, the same time” (p. 13), a process he terms synchronous convergence. Memory is associative and acquired through the senses, consisting of neuronal networks formed through this convergence [2,19,20]. To understand how this networked architecture operates over time, memory can be broken down into three distinct, interrelated processes. First, memories have to be formed (encoding). Second, memories must be maintained over time (storage). While they are stored, they may undergo consolidation or strengthening. Finally, for a memory to be used, it needs to be accessed (retrieval) [21].

Through repetition, these associative networks support stable conceptual representations and rule-guided, goal-directed action [2,19,22,23,24,25].

Neuronal networks are widely distributed across sensory and motor areas [22,23,26,27]. Sensory areas specialise in perception and acquire perceptual (declarative) memory, including concepts and facts, through repetition. For example, the concept “bird” is represented in a neuronal network as a composite that associates smaller networks encoding visual features (e.g., shape and colour), a name (word), foreign names, and other relevant information through context-based experience [2,19,20]. Within these networks, Wernicke’s area supports object naming and comprehension of spoken and written words and sentences [2,28].

Conversely, motor areas—primarily the prefrontal cortex and the basal ganglia—control actions like speech and support the acquisition of motor (procedural) memory, including conditional IF–THEN rules and motor programmes, through repeated practice. Rules—such as IF stimulus A, THEN select response X to achieve Y—are learned through context-based experience, linking perceptual (e.g., visual and auditory) cues to goal-directed responses (actions). For example: IF a traffic light turns red, THEN stop the car. These conditional regularities allow behaviour to adapt to context rather than relying on rigid stimulus–response links [2,20,23,24,25,29,30,31].

Producing goal-directed action, including meaningful speech, requires a motor programme or action schema. As conceptualised by Lashley [1], this integrative, general-domain schema consists of groups of words or motor acts and specifies the hierarchical planning (syntax) and linear production (motor output) of the constituents of a sequence. Fuster [2,23] adds that these programmes are also organised in accordance with rules. For further review, see Lashley [1], Chomsky [32] and Fuster [2,19,23], as cited in Mendoza and Zulueta [33]. Both the prefrontal cortex (located in the frontal lobe) and the basal ganglia (situated deep subcortically) exert hierarchical syntax [19,23,34]. In syntactic construction, the prefrontal cortex acts as the highest organiser of language and grammar, dynamically interacting with networks that provide the lexicon. Whenever attention and deliberation are needed, networks in the prefrontal cortex exert the act-by-act or word-by-word planning and production of sequences. Prefrontal duties exclude overlearned actions, routine and unimaginative language, and familiar rules and their implementation. The prefrontal cortex is involved in early learning, but once established, these old activities require the basal ganglia [2,19,20,23,27]. With sufficient repetition and practice, the basal ganglia segment sequences—like when we chunk a phone number for easier learning—and acquire motor chunks. This area automatically plans and produces sequences using these clusters [3,35,36,37,38] (see Figure 3).

Mendoza and Zulueta [33] applied this neurobiological chunking mechanism to classroom research on children acquiring English as a Foreign Language (EFL). This study tested a teaching framework that employed multimodal sensory chunks to offset ‘the problem of cognitive overload’ and promote learning and automatic speech. The pedagogy used cross-domain input that combined visual context with language, specifically illustrations paired with chunk-based sentences (e.g., [He plays] [a lot]). Sensory chunking is a technique that consists in teaching with sentences constructed entirely from language blocks that can be perceived through the senses, e.g., vision, audition, and touch—hence the term sensory chunks. This central technique was refined with three complementary strategies: incremental sentence length, chunk reutilisation and recombination, and sentence pattern iteration.

The framework integrated additional core principles. First, repetition promotes memory formation and strengthening, which applies to both perceptual and motor memories. Such a cognitive operation involves repeated exposure to specific sensory configurations and repeated action practice [2,33]. This premise was exemplified by techniques such as sensorimotor rehearsal and motor rehearsal. For example, sensorimotor rehearsal trains learners to translate perception (visual context) into action (speech) using images and chunked sentences that can be read and heard and are used for oral rehearsal. Such a technique was inspired by the neural processes of Long-Term Potentiation and sensorimotor integration, which is discussed later in this section. Second, information which cannot be retrieved cannot be used. This tenet was operationalised through rehearsed information testing, which encourages learners to actively retrieve previously practised content to forge stronger memories—ultimately driven by the synaptic consolidation of Long-Term Potentiation—and prevent decay. More precisely, this technique seeks to mitigate ‘the problem of inattentional blindness’ and promote learning [33]. It was informed by research on the cognitive functions of attention and memory, including the issues of retrieval practice and the testing effect [39,40,41,42] (See Section 4.3 for further discussion on these two principles).

The effectiveness of this pedagogy was evaluated using an oral sentence transformation task. Although reaction times were not measured, the experimental groups achieved higher accuracy rates than the control group for sentence segments containing the third-person singular -s (e.g., ‘he VERB + s’ as in ‘he plays’). This suggests that this framework successfully facilitated learning, presumably by mitigating cognitive load. Together, the principles established across this literature review provide the broader empirical and theoretical foundation for the computational model examined here. In the present study, we focus on the learning of words and conditional IF–THEN rules (specifically the Mutual Exclusivity rule) in Large Language Models. However, as detailed in our conclusions, our research agenda will expand to three core investigations: first, exploring word learning through ambiguous sentences (see Section 1.4); second, examining rule learning and concrete syntactic structures by injecting conditional IF–THEN rules into the system; and third, applying sensory chunks as visual input to drive the acquisition of syntactic proficiency and rule abstraction.

At the neural level, sensory and motor regions work together in sensorimotor integration—the process through which perceptual input is transformed into goal-directed action. Humans interact with their environment by efficiently collecting and processing sensory information from multiple modalities, e.g., vision and hearing [2,23,43]. Humans are predominantly visual: perceptions, movements, and social behaviour rely largely on visual cues. The brain requires context—particularly visuospatial information—to assign meaning to objects and events, and guide action [44,45]. Understanding how the brain organises context-based information to plan and execute actions flexibly and adaptively is essential for learning [2,23,43,46].

The prefrontal cortex carries out sensorimotor integrations to plan and produce sequences at the initial stages of learning and whenever sustained attention to every item in a sequence is required [2,23]. With sufficient practice, the basal ganglia perform these integrative operations to plan and produce well-rehearsed, automatic sequences, including routine speech. The basal ganglia, mainly via their main input nucleus, the striatum, receive inputs from broad areas of the cerebral cortex and project back mainly to the frontal lobe. These connections are often referred to as cortico-basal ganglia loops. Long-Term Potentiation has been demonstrated in cortico-striatal pathways. Cortico-striatal synapses are thought to be sites of neuroplasticity, emphasising the role played by the basal ganglia in learning and memory [36,47,48]. In a novel situation, neurons in the basal ganglia are activated only weakly because that particular context is not associated with a specific behavioural pattern. However, in highly familiar environments, neurons in the prefrontal cortex signal the basal ganglia, which release the automatic action [49].

Together, these neuroscientific findings and their empirical applications suggest that learning emerges from the interaction between sensory and motor systems, driven by the repetition-based strengthening of synaptic connections—a cellular process known as Long-Term Potentiation. Sensory areas organise perceptual input into categories, whereas motor areas apply conditional IF–THEN rules linking perceptual information to produce purposeful, rule-guided action, including meaningful speech. In language learning, these mechanisms support the formation of category–word mappings and conditional IF–THEN rules for flexible, adaptive use.

These neuroscientific accounts motivated the theoretical framework and experimental design examined here. The Fictitious-Animal Paradigm evaluates whether the Large Language Model can store perceptual categories (C₁), store category–word mappings (C₁ == W₁), store the Mutual Exclusivity rule—whereby an already labelled category cannot receive a novel word (W₂), instanced formally as ‘IF C₁ == W₁ → THEN C₁ != W₂’—and retrieve stored information to assign novel labels through inferential reasoning. How these rule-guided, conditional structures are formalised as logical operations outside organic substrates is a foundational inquiry in cognitive psychology.

1.2. Cognitive Psychology: Conditional IF–THEN Rules

To understand how linguistic experience transforms into computational constraints, cognitive science has long relied on the formalisation of rule-based symbolic architectures. Classic cognitive architectures, such as Anderson’s [50] ACT-R (Adaptive Control of Thought—Rational), formalised human cognition through production systems driven by conditional rules. Within this framework, cognitive operations—including language production—follow explicit structures: IF ⟨condition⟩ → THEN ⟨action⟩. For example, ACT-R models generative syntax operations as: IF the goal is to generate the plural of a noun, THEN say “noun + s” (p. 373). Here, the IF clause outlines the specific contextual prerequisites, while the THEN clause defines the subsequent action. In human learners, these rules initially function as explicit declarative guides before becoming proceduralised and automatised through repeated practice.

However, traditional symbolic architectures typically process these rules in isolation from sensory environments. As Thagard [25] notes, while systems like ACT-R can incorporate external inputs, they generally translate rich perceptual data into abstract symbolic verbal forms, discarding the original sensory complexity. Thagard underscores that robust conditional and referential learning depends fundamentally on integrating non-linguistic cues directly with linguistic input, rather than relying on disembodied symbolic manipulation.

Furthermore, research in cognitive psychology demonstrates that human conditional reasoning improves significantly when logical premises operate over well-defined conceptual categories. Markovits and Vachon [51] showed that children perform poorly on conditional tasks when the underlying conceptual representations are unstable or arbitrary, such as in the pseudo-conditional premise: “If one fretres, the puyge will fall”. Their findings indicate that reasoning about IF–THEN relationships depends strictly on the availability of clearly structured conceptual categories, which provide a meaningful baseline for inference.

To isolate the role of categorical support, Markovits et al. [52] distinguished between structure-based and non-structured conditionals, defined by “IF [CATEGORY] → THEN [PROPERTY]” and meaningless premises, respectively.

Structure-based conditionals involved premises where the antecedent and consequent share an implied relationship. This group included imaginary categorical conditionals, which conveyed a category–property relationship using invented terms (e.g., “If a plant is a mandola [CATEGORY], then it has blue leaves [PROPERTY]”; “If an animal is a bori, then it has red wings”). Additionally, this group also encompassed familiar categorical conditionals that rely on concrete, meaningful category–property links, such as “If a plant is a cactus, then it will have thorns” and “If an object is a knife, then it has a blade.”

Non-structured conditionals encompass statements in which the antecedent and consequent are arbitrary or meaningless. This group covers basic conditionals, which use familiar terms but have an arbitrary connection, for instance: “If a circle is red, then the star is black” and “If Melinda wears a red sweater, then she will wear green pants.” It further comprises abstract premises, which employ invented terms with no semantic or categorical support, requiring purely formal processing. Examples include: “If there is a flop, then there is a mauchard” and “If there is a zurde, then there is a triffart.”

According to Markovits et al. [52], the results revealed that human participants processed structure-based conditionals significantly faster and more accurately—even when dealing with entirely fictitious terms—because the category-property structure provides a meaningful inferential link. This demonstrated that both the concreteness of the categories and the meaningfulness of the relationship between antecedent and consequent enhance conditional reasoning. These findings highlighted an important principle for the present study: conditional reasoning operates most effectively when linguistic expressions refer to stable conceptual categories organised within structured conditional statements.

Consequently, the experimental design crafted by Markovits et al. [52] directly addresses the limitations of traditional systems, such as ACT-R, criticised by Thagard [25]. While legacy models often discard sensory richness by relying on decontextualised theoretical rule representations that operate independently of perceptual context, the proposed theoretical framework integrates perceptual complexity into conditional logic. To test this setup (see Section 2), a Fictitious-Animal Paradigm was used. The authors employed fictitious creatures (e.g., flug, gint) with specific visual features to neutralise pre-existing real-world biases and measure genuine inferential reasoning. To further validate our proposal, the next section examines how this theoretical framework is rooted in developmental psycholinguistics and usage-based approaches, specifically exploring how children acquire words, syntactic patterns, and rules.

1.3. Developmental Psychology and Usage-Based Approaches: Category-Word Mapping, Rule Abstraction, and Syntactic Frames

Developmental research demonstrates that vocabulary acquisition is initially anchored in contexts of joint attention, where caregivers and children coordinate their focus toward a shared object. For example, if a child points to an apple while a parent says “That’s an apple”, this shared focus helps the child understand and learn the new word [53,54,55]. This process aligns with Fuster’s [19] neurological account of associative learning and synchronous convergence described above.

Beyond associative mapping, children exploit the Mutual Exclusivity rule. Children assume that each category possesses a unique linguistic label: “A single object cannot both be a cow and a bird or a dog. Thus, for categories to be informative about objects, they will tend to be mutually exclusive” [4] (p. 123). Thus, “if a child already knows a label for an object, a new label for that object should be rejected” [4] (p. 121). In their classic disambiguation task, Markman and Wachtel [4] presented children with pairs of objects—one familiar and one unfamiliar—and asked them to “Show me the X”. Children systematically selected a novel referent when hearing a novel label, rejecting the familiar category in favour of the unlabelled one.

The origin of this Mutual Exclusivity rule remains a cornerstone debate within cognitive science, broadly polarised into two major theoretical perspectives. On the one hand, the Nativist Framework argues that, without internal constraints, a novel label could logically refer to an infinite array of non-accidental properties, such as a particular colour or texture. Thus, children are born with a structural constraint or a built-in rule specifying that each object can have a unique label [4]. This innate blueprint is designed to bypass Quine’s [56] problem of referent indeterminacy. This dilemma establishes that for any given ostensive pairing—such as an adult uttering a novel word while pointing toward a complex visual scene—an indefinite number of logically valid, alternative hypotheses remain perfectly consistent with the input. On the other hand, the Emergentist and Usage-Based Framework contends that mutual exclusivity is not an innate rule. Instead, evidence shows this constraint is actively extracted from regularities in communicative experience and developed as an emergent property of domain-general cognitive systems, namely attentional and memory mechanics [57,58,59].

The authors bridged this gap by formalising the Mutual Exclusivity constraint as an operational conditional IF–THEN rule:

IF C₁ == W₁ → THEN C₁ != W₂

where “IF an object or perceptual category (C₁) is already associated with a word (W₁), THEN it cannot receive a new word (W₂).” This rule is an integral component of the proposed theoretical framework, which establishes the necessary milestones for the acquisition of vocabulary and linguistic motor programmes. This includes learning the Mutual Exclusivity rule, as well as the rules of syntax and grammar, as conditional IF–THEN statements. Specifically, four developmental milestones, which reflect four core human cognitive capacities, were translated into four testable computational capacities applied to word learning in Large Language Models (see Section 2).

Additionally, developmental psychology and usage-based approaches emphasise that as linguistic experience scales, learners extract recurrent regularities and patterned frames, such as the constructional frame “This is a X”. Through repeated exposure involving observable contexts paired with linguistic input, children extract these frames as abstract representations that can be generalised to novel exemplars [57,60,61,62,63]. Classic work using artificial lexicons demonstrates that children can learn abstract rules from sentence-level input. Shaffer and Kipp [64] reported that Berko [65] investigated children’s understanding of the rules for forming plurals in English using the well-known “wug” experiment. In this paradigm, children were presented with illustrations of fictitious animals and were prompted with sentences such as “This is a wug. Now there are two ___.” Most children systematically responded “wugs,” demonstrating their capacity to apply the English plural rule to unfamiliar lexical items (see Figure 4, using the word “nat” instead of “wug”).

Robust evidence for this rule abstraction is also found in early overregularisation errors, such as producing “I brushed my tooths”, “She goed”, or “It runned away”, which reflect a systematic application of grammatical rules [66,67,68].

In summary, children’s patterns reflect a progression from simple mappings toward the acquisition of syntax and grammar. However, unlike Anderson [38], who describes rules as decontextualised theoretical rule representations (e.g., IF the goal is to generate the plural of a noun, THEN say “noun + s”; p. 373), the authors propose that conditional logic relies on context-based experience, where structured cross-domain input is of paramount importance. By integrating both visual context and linguistic stimuli, the model can perform inference grounded in current visual contexts.

1.4. Visual Context and Linguistic Stimuli: Acquiring Meaningful Language

While the previous sections have established the neural, cognitive, and developmental foundations of associative learning and IF–THEN rule formation, this section focuses on how the integration of visual context and language (here, written text)—acting as structured cross-domain input—enables inferential reasoning for word learning and, overall, meaningful language acquisition.

In natural settings, language acquisition involves constructing meaning through exposure to complementary non-linguistic and linguistic information in multimodal contexts (see Section 1.1). Within these environments, meaning originates primarily from cross-domain input, where visual scenes ground the accompanying linguistic stimuli.

The role of visual context becomes particularly important under conditions of referential ambiguity, where multiple objects may correspond to a given expression. In such cases, interpretation requires selecting a mapping that satisfies both stored category–word associations and the constraints imposed by the current visual scene. Visual context thus narrows the space of possible interpretations and enables inference by eliminating incompatible alternatives rather than through direct selection alone. In this sense, visual context does not merely accompany linguistic input, but constrains how both words and linguistic structure are learned and interpreted.

However, traditional generativist approaches often emphasise linguistic input alone. According to Chomsky’s [69] generative framework, syntactic comprehension depends on hierarchical phrase structure, often illustrated through syntactic trees [70]. For example, the sentence “The nat is looking at the pog with the telescope” contains structural ambiguity (see Figure 5).

While abstract syntactic representations capture formal dependencies [69], they do not specify how ambiguity is resolved during online comprehension in perceptually rich environments. In contrast, real-time language understanding relies on integrating visual context with linguistic input. Children interpret language in rich perceptual contexts that constrain meaning and resolve ambiguities. Thus, interpretation does not arise from abstract syntactic structures alone, nor from purely symbolic rule systems as in classical cognitive architectures, such as those described in Section 1.2 [50], but from their interaction with perceptual and contextual information. Visual context, therefore, plays a decisive role in disambiguating linguistic input and guiding inferential reasoning to resolve referential ambiguity.

Successful comprehension and conditional reasoning, therefore, depend on integrating cross-modal information, with visual context playing a primary role. Visual context is crucial in grounding linguistic input in observable entities and actions, supporting both word meaning assignment and syntactic structure interpretation in real-world settings. This anchoring is particularly important given the prominence of vision in human perception and the broader role of sensorimotor integration—the translation of perception into action (see Section 1.1).

Figure 6 illustrates this interaction between visual context and linguistic stimuli using novel lexical items. While the syntax is ambiguous, the accompanying scene—the visual context—constrains which interpretation is compatible with the observed situation. In this way, visual context guides both category-word mapping and the inferential processes required to interpret linguistic expressions relative to the environment.

2. Milestones and Capacities: A New Framework

To understand how cross-modal input—specifically, the pairing of visual context and linguistic data—is progressively organised into conditional IF–THEN rules that drive inferential reasoning, the authors propose a cognitive architecture based on four developmental milestones in human language acquisition. Together, these milestones model how vocabulary and linguistic motor programmes are acquired, which includes the learning of the Mutual Exclusivity rule, as well as the rules of syntax and grammar, as conditional IF–THEN statements:

Milestone 1: Storing perceptual categories. Learners detect visual features (e.g., those of the nat) and organise them into perceptual categories, which serve as representational classes containing multiple referents. This step provides the categorical substrate for subsequent lexical mapping. For example, repeated exposure to visually similar instances of a novel creature allows the learner to form a stable category representing that entity.

Milestone 2: Storing category-word mappings. The co-occurrence of perceptual categories with linguistic input allows learners to associate novel labels with specific referents and categories through repeated exposure. For instance, sentence frames such as “This is a X” (e.g., “This is a nat”) support the assignment of the label “nat” to an object instance within its category.

Milestone 3: Storing conditional IF–THEN constraints. Learners register the constraints governing label assignment across categories. The authors conceptualise the Mutual Exclusivity rules as an operational conditional IF–THEN rule: “IF an object or perceptual category (C₁) is already associated with a word (W₁), THEN it cannot receive a new word (W₂).” In our architecture, storing this conditional constraint is what enables the system to resolve referential ambiguity when encountering unfamiliar items.

This milestone also accounts for the registration of the rules of syntax and grammar as higher-order conditional IF–THEN statements, such as:

• IF a novel referent is introduced, THEN use the frame “This is a X”
• IF this referent is presented alongside an identical counterpart, THEN use the frame “These are NUMBER(two) X + s”
• IF a referent is performing an action, THEN use the frame “The X is VERB + ing”

Milestone 4: Deploying inferential reasoning. Stored conditional IF–THEN rules are retrieved and applied to resolve referential ambiguity for word learning and to build the linguistic motor programmes governed by the prefrontal cortex.

Regarding word acquisition, if one object has already been labelled nat, and the learner is asked “Which one is the pog?”, the learner retrieves and applies the stored instance of the Mutual Exclusivity rule to resolve the ambiguity. During inference, the previously labelled entity is excluded, and the novel label pog is assigned to the unlabelled creature. As a consequence, referential ambiguity is resolved, which facilitates word learning.

Concerning linguistic motor programmes, inferential reasoning enables the formulation of action schemas that require sustained attention and deliberation. As described in Section 1.1, the neurobiological execution of these schemas depends entirely on the level of practice. Initially, these actions rely on the prefrontal cortex, which exerts an attentive, deliberative, word-by-word planning and production of the sequence. As these patterns become automated through practice, processing shifts to the basal ganglia. This motor region acquires motor chunks through repetition, facilitating automatic speech.

As suggested by Mendoza and Zulueta [33], when processed as a well-rehearsed chunk, the grammatical rule is automatically integrated into that memory unit (e.g., [he play + s]). This aligns with Fuster’s [23] assertion that the prefrontal cortex is not responsible for ‘overlearned actions, familiar rules and their implementation, as well as routine and unimaginative language’ (p. 253). With sufficient repetition, the basal ganglia take over, bypassing the need for any deliberative cognitive computation [2,19,20,23,27].

In this view, these four milestones define a clear architecture: they illustrate how structured cross-domain input gives rise to conditional IF–THEN rules, how these rules enable both inferential reasoning and the attentive formulation of linguistic motor programmes, and how continuous practice ultimately yields automatic speech. However, the present study examines whether structured cross-domain input can generate conditional IF–THEN rules, ultimately enabling adaptive, context-driven inferential reasoning for word learning (see Figure 7).

Our theoretical model delineates four developmental milestones (M1–M4) in human language acquisition, which are translated into four cognitive capacities (K1–K4). This architecture constitutes the full potential of this learning framework, ranging from simple perceptual categorisation to refined, chunk-based syntax and grammar. In this framework, the milestones (M1–M4) establish the developmental “route”, while the capacities (K1–K4) are activated, prioritised, or omitted depending on the specific requirements of the learning context. This operational flexibility implies that the acquisitional process does not follow a rigid path. For example, in certain natural language settings, word learning may only involve K1 and K2—corresponding to Hebbian [14] learning principles, Fuster’s model of synchronous convergence [2,19,20], and joint attention in word learning [53,54,55]. However, when the acquisition of words requires resolving referential ambiguity, specific foundational capacities—namely, K1–K3—need to be stored and retrieved for the execution of inferential reasoning (K4).

Broadly, the four capacities for word learning encompass: (1) the storage of perceptual categories (K1), (2) the storage of category–word mappings (K2), (3) the registration of the Mutual Exclusivity rule (K3), and (4) the deployment of inferential reasoning (K4), involving the retrieval and application of such rules to resolve referential ambiguity. As these capacities are practised, they facilitate memory consolidation.

Similarly, the acquisition of syntax and grammar involves (1) the storage of perceptual categories (K1), (2) the storage of category–word mappings (K2), (3) the storage of higher-order conditional IF–THEN statements (K3), and (4) the deployment of inferential reasoning through the retrieval and application of these higher-order rules (K4). With sufficient repetition, these capacities enable the acquisition of linguistic motor programmes that comply with rules. In sum, the authors propose a unified theoretical framework in which this four-stage setup (K1–K4) serves as the pathway for lexical, syntactic, and grammatical development.

Crucially, these four cognitive capacities inspired the four computational capacities tested in the present study. Specifically, these computational capacities (K1–K4) were evaluated using a Fictitious-Animal Paradigm, which assessed the system’s ability to transition from perceptual categorisation to inferential reasoning in word learning. This paradigm also allowed for the isolation of pre-existing real-world biases and made it possible to measure whether the ability for flexible, context-driven inference stems from structured cross-domain input: complementary non-linguistic and linguistic stimuli (visual context and written text).

Computational Capacities Operationalised in the Present Experiment

A Fictitious-Animal Paradigm operationalised the four computational capacities (Ks) for word learning outlined at the beginning of this section [71]. This experimental framework employed a dataset of 32 unique creatures and 32 paired pseudowords (such as flug or gint) that allowed the authors to track the model’s performance across the four computational capacities (Ks) applied to word learning: storing perceptual categories (K1), storing category–word mappings (K2), storing the Mutual Exclusivity rule (K3), and deploying inferential reasoning (K4). This setup, grounded in the current visual context, enabled the system to engage in adaptive, flexible word learning.

Methodologically, the Fictitious-Animal Paradigm focused on single exemplars that instantiated distinct categories (where a category constituted the set of visual features and words such as flug or gint served as category labels). While the first two computational capacities were labelled as storing a perceptual category (K1) and a category-word mapping (K2), the authors acknowledge that the proposed setup worked under a single-exemplar training regime. However, this condition acts as the necessary operational transition toward full categorical abstraction, which will be expanded in future work with multiple varied exemplars per category.

Capacity 1: Storing a perceptual category. The model detects the visual features of a single creature and clusters them into a stable representation, which serves as the representational basis for a category (C₁). Essentially, the model stores these visual features as a foundational base before linking them to any linguistic label. Evidence from neuroscience [2,19,20] and developmental research [60,65] suggests that clustering sensory input into stable referents provides the necessary representational substrate to build meaningful categories onto which lexical items can later be mapped.

Capacity 2: Storing a category-word mapping. The model associates the perceived entity with a novel lexical label (e.g., “flug”): it maps the co-occurrence between the perceptual features and the linguistic input. This process functions as the representational substrate of a category-word mapping (C₁ == W₁). Such binding mechanisms are widely discussed in the literature on early word learning [53,54,55] and research on associative neural networks [2,14,19,20].

Capacity 3: Storing the Mutual Exclusivity Rule. The model is presented with a second visual category and a second pseudoword and must register and store the Mutual Exclusivity rule as a set of conditional IF–THEN statements:

General rule formalisation:
IF C₁ == W₁ → THEN C₁ != W₂
Instantiation for the familiar creature:
IF C₁ == flug → THEN exclude C₁
Instantiation for the novel creature:
IF C₁ == flug → THEN C₁ != gint

This stage represents the model’s ability to register and store these logical constraints that prevent incorrect label assignments. This capacity structures the lexicon by defining which visual category-word pairings remain permissible and which the model must exclude based on the current visual scene, directly enabling subsequent inferential processing. Capacity 3 stores these constraints, while Capacity 4 retrieves and executes them. Developmental psycholinguistics extensively documents this Mutual Exclusivity rule as a core mechanism that guides referential mapping [4]. This approach aligns with usage-based theories, where rules emerge from the communicative environment rather than from formal theoretical abstractions. See also neuroscientific research on neuronal networks, sensorimotor integration, motor programmes, and rules in Section 1.1. For the current experiment, rather than formalising the Mutual Exclusivity rule as a theoretical abstraction [50,69], the authors inserted a context-dependent constraint that prevented the model from assigning a novel label (e.g., gint) to a familiar creature (e.g., flug). Section 4.3 provides a detailed discussion of how this restriction guided the model’s inferential reasoning.

Capacity 4: Inferential reasoning. The model must perform referential disambiguation, guided by the current visual context (the two-animal scene). To do so, the model must actively retrieve three discrete components from storage (K1–K3): a specific perceptual category (K1), its corresponding category-word mapping (K2), and the complete set of conditional IF-THEN rules (K3). Applying this retrieved K3 information during inference in response to the current visual scene transforms static, stored data into flexible, adaptive conditional resolutions:

General rule formalisation:
IF C₁ == W₁ → THEN C₁ != W₂ → Resolution: C₂ == gint
Instantiation for the familiar creature:
IF C₁ == flug → THEN exclude C₁
Instantiation for the novel creature:
IF C₁ == flug → THEN C₁ != gint → THEREFORE C₂ == gint

Thus, the model excludes C₁ and assigns W₂ (gint) to C₂. The model applies this set of logical constraints, deriving the correct new mapping by inference when it contrasts a novel animal and a novel pseudoword with the previously stored category-word mapping. This mirrors developmental studies on exclusion-based reasoning and aligns with computational architectures where structural rules guide inference. Furthermore, neurobiological evidence supports conditional logic through context-dependent IF-THEN action selection in the brain (see Figure 8).

3. Materials and Methods

The model used for this exercise was implemented using the BLIP-2 (Bootstrapping Language-Image Pre-training) framework, a system to efficiently pair visual context and textual information. BLIP-2 combines three main components: an Image Encoder, a Querying Transformer (Q-Former), and a Large Language Model. This modular design enables the integration of pre-trained vision and language models, significantly reducing computational training requirements while facilitating cross-domain knowledge transfer between non-linguistic and linguistic inputs. To satisfy software reporting requirements and ensure full reproducibility, the technical specifications of the core implementation model are detailed in

Table 1. Software details.

Software/Model	BLIP-2, Salesforce/blip2-opt-2.7b
Developer Company	Salesforce.com, Inc./Salesforce AI Research
Address	Salesforce Tower, 415 Mission Street, 3rd Floor, San Francisco, CA, 94150, USA
Platform/Repository	Hugging Face
Programming environment	Python 3.14.5

.

This section focuses on three core aspects: BLIP-2 components, Loss Functions, and Attention Mechanisms. The visual processing stage is performed by a Vision Transformer (ViT)-based Image Encoder, which converts input images into dimensional feature representations (embeddings) capable of capturing both semantic and spatial information. These visual features are subsequently processed by the Q-Former, a Transformer-based module that acts as an intermediary between the visual scenes and language components.

By using learnable query tokens alongside self-attention and cross-attention mechanisms, the Q-Former selectively extracts and compresses the most relevant information from the image into a compact semantic representation. Finally, the resulting visual embeddings are provided to the Large Language Model, which integrates them with the textual input to generate contextually coherent outputs. This architecture allows the system to perform cross-domain tasks by jointly reasoning over non-linguistic and linguistic inputs.

Beyond its three main architectural components, BLIP-2 relies on several key learning objectives and attention mechanisms that enable effective alignment between textual and visual information. During the first stage of pre-training, the Q-Former is optimised using three complementary loss functions: Image-Text Contrastive (ITC) Loss, Image-Text Matching (ITM) Loss, and Language Modelling (LM) Loss. These objectives jointly improve the model’s ability to process and generate cross-domain content.

The ITC Loss aligns image and text representations within a shared embedding space. The model maximises the similarity between corresponding image-text pairs while minimising similarity with non-matching pairs. This encourages the extraction of semantic features that are meaningful across domains, improving retrieval and recognition capabilities.

The ITM Loss complements the contrastive objective by performing a binary classification task that determines whether a given image and text pair are related. The Q-Former uses its query embeddings to aggregate visual information, and a classification head predicts whether the pair is matched or mismatched. Hard negative samples are often incorporated during training to increase robustness and discrimination performance.

The third objective, LM Loss, enables generative learning. Given visual features extracted from an image, the model is trained to predict the next token in a text sequence using an autoregressive formulation. This objective allows the Large Language Model to generate image-grounded descriptions, answers, or instructions while maintaining linguistic fluency and coherence.

Another crucial aspect of BLIP-2 is the strategic use of attention mechanisms. Within the Image Encoder, the Vision Transformer (ViT) employs self-attention, allowing each image patch to interact with all other patches and capture long-range spatial dependencies. The Q-Former combines both self-attention and cross-attention layers. Self-attention enables communication among the learnable query tokens, whereas cross-attention allows these queries to attend to the visual embeddings produced by the Image Encoder. Through this process, the Q-Former compresses high-dimensional visual information into a compact set of semantic representations.

The Large Language Model also relies on specific attention mechanisms to execute the final integration. It uses causal self-attention, which ensures that each generated token depends only on previously generated tokens and enables autoregressive text generation. Additionally, cross-attention layers integrate the visual representations produced by the Q-Former into the language generation process. This interaction allows the Large Language Model to condition its outputs on image content while preserving the contextual understanding acquired during large-scale language pre-training.

The authors chose not to use alternative models such as LLaVA and Qwen2-VL because previous studies have addressed the same task and did not demonstrate significant differences in their results. To support this conclusion, the authors reference the following studies: Liu et al. [72], Xiong et al. [73], Li et al. [74], and Wang et al. [75].

4. Experimental Design

The primary objective of this experimental design was to investigate whether Large Language Models can display human-like inferential reasoning in word learning when exposed to structured cross-domain input. Specifically, this framework assessed how a model processes complementary non-linguistic input (visual context) and linguistic input (written text) to accomplish four tasks: storing a perceptual category (K1), storing its corresponding category–word mapping (K2), storing the Mutual Exclusivity constraint as a logical architecture (K3) when presented with a novel creature, and performing inference (K4). In addition, rather than relying on abstract symbolic rules, the framework implemented the constraint (K3) as a context-based constraint (a restriction preventing the assignment of a novel label Y to a familiar creature X). Thus, to rigorously assess this process, the experimental workflow evaluated the model’s ability to integrate the four computational capacities and to store and adaptively manipulate formal variables (e.g., label X and category Y) across different visual contexts.

The experiment presents a controlled Fictitious-Animal Paradigm featuring 32 creatures and 32 pseudowords, divided into two phases: single-animal scenes and dual-animal scenes (visual contexts). The single-animal scene phase entailed storing categories and mappings (K1–K2). The dual-animal scene phase required retrieving K1–K2, storing logical constraints (K3), and retrieving all stored data (K1–K3) to deploy inferential reasoning (K4). Scenes with two animals created the necessary referential ambiguity to facilitate rule training and assess whether the model could apply the Mutual Exclusivity rule using the context-dependent constraints to resolve novel mappings.

4.1. Dataset Characteristics

The dataset comprised 32 fictitious animals and 32 corresponding pseudowords. Each creature was defined by a unique combination of visual attributes, including variations in colour, shape, and body morphology. To increase the challenge to the model’s discriminative capabilities, several creatures shared highly similar visual features, preventing successful mapping through simple visual patterns. To maintain strict dataset consistency, all images were normalised to 364 × 364 pixels. The creatures were placed against a uniform, light-coloured background to facilitate the extraction of relevant visual features.

The database consisted of 16 fictitious animals used during the single-animal phase and 32 creatures in the dual-animal phase. This second phase included the 16 animals from the previous phase and 16 new creatures and involved 256 pairs of fictitious animals. By rearranging the position of these animals, the authors generated 512 combinations. Out of these 512 combinations, 32 were used for inference, and 1 was set aside for integrity code testing. Consequently, to rigorously evaluate our computational model, the dual-animal phase included 479 images and 958 corresponding annotations. Each dual scene received two annotations: one for the familiar creature and another for the novel creature. A larger dataset was deemed unnecessary, as a pre-trained neural network can address general tasks effectively.

4.2. Phases and Computational Capacities

As mentioned, the experimental setup was divided into two phases: single-animal and dual-animal contexts. K1 and K2 operated during the single-animal phase. In contrast, K3 and K4 functioned in the dual-animal phase, since both rule acquisition and inference via exclusion require a context that enables comparison between two entities.

4.2.1. Phase 1: Single-Animal Scenes (K1–K2)

The first phase used single-animal visual contexts to establish data storage within the model. For K1, the model processed non-linguistic stimuli—using five specific prompts per exemplar (e.g., “Can you describe the appearance of this imaginary creature?”, “What is it called?”, etc.)—to cluster visual features into a distinct perceptual category (C₁). Subsequently, for K2, the model mapped the perceptual category (C₁) to a specific lexical item or pseudoword (W₁), establishing a category-word mapping (C₁ == W₁). This phase utilised 16 images, with each image associated with five questions, resulting in a total of 80 annotations. Figure 9 illustrates a prompt used during this phase. Elements enclosed in square brackets—colour-coded in blue—denote dynamic variables representing specific visual features and category labels (pseudowords) applied to each category exemplar.

4.2.2. Phase 2: Dual-Animal Scenes (K3–K4)

The second phase introduced a two-animal visual scene containing a familiar animal from the single-animal scene and a novel creature. The model was tasked with retrieving previously stored information (K1–K2) and storing the Mutual Exclusivity rule (K3). For K3, the authors established a context-dependent constraint, ensuring that the system did not assign a novel label (e.g., gint) to a familiar creature already designated by its known label (e.g., flug). Additionally, for K4, the model needed to retrieve all stored data (K1–K3) to perform inferential reasoning and resolve referential ambiguity (K4).

Out of the generated combinations, 32 were used for inference, and 1 was set aside for integrity code testing, leaving 479 images featuring two creatures. Because a question was asked for each creature’s position in the image (with the novel and familiar creatures appearing in both Position A and Position B), the number of annotations was doubled, resulting in 479 images paired with 958 associated annotations.

To ensure methodological robustness and prevent bias, the authors administered four randomised queries for each dual-animal scene, balancing both the target identity (the familiar creature, “Which one is the flug?”, or the novel creature, “Which one is the gint?”) and its spatial location: each creature was queried once in Position A and once in Position B. This safety protocol ensured that the model’s performance relied exclusively on active inferential reasoning and did not exploit experimental patterns.

Moreover, dual-animal scenes combined familiar animals with entirely unknown ones. This protocol prevented the model from depending on previously learned associations when faced with referential ambiguity. Instead, it required the model to engage in adaptive, real-time reasoning based on the current visual context. By ensuring that novel entities received significantly less training than familiar ones, the authors also established a rigorous protocol to test the model’s capacity for inference under conditions of representational instability. Figure 10, Figure 11, Figure 12 and Figure 13 illustrate the four queries administered during the dual-animal phase. Elements enclosed in square brackets—colour-coded in blue—denote dynamic variables representing specific visual features, category labels (pseudowords) applied to each category exemplar, and the target creature’s position (A/B).

4.3. Capacities, Variables, and Meaningful Context-Based Experience

A critical methodological consideration in evaluating inferential reasoning is whether the model’s performance stems from autonomous inferential reasoning or merely from adherence to an abstract rule or static template. Injecting a context-based constraint (such as “I must not assign the novel label Y to the familiar creature X”) does not mean providing the model with a pre-packaged answer to the question “Which one is the X?” (see Figure 10, Figure 11, Figure 12 and Figure 13). Instead, this constraint functions as a “negative hypothesis” (what the model must not do). This restriction or prohibition—for example, instantiated as “I must not assign the novel label gint to the familiar creature flug”—initiates the entire logical mechanism: the model discards the familiar creature, isolates the remaining perceptual category, and maps the novel referent by exclusion.

This constraint constitutes a conditional IF-THEN rule (IF C₁ == W₁ → THEN C₁ != W₂) that forces the systematic use of variables involving the fictitious animals in response to the two-animal scene. Here, the model discards the familiar creature and performs dynamic, real-time, context-based inferential reasoning: first, excluding the familiar visual referent (IF C₁ == flug → THEN exclude C₁) and subsequently, assigning the novel label to the novel perceptual category through inferential reasoning (IF C₁ == flug → THEN C₁ != gint, THEREFORE C₂ == gint).

Moreover, cross-domain input (visual context and written text) might enable a functional analogue to meaningful, context-based learning that deploys the computational capacities (K1–K4) as inspired by the developmental milestones (M1–M4) and cognitive capacities (K1–K4) attributed to humans in our theoretical framework. Crucially, rather than relying on decontextualised and meaningless theoretical abstractions (see Anderson [50]; Chomsky [69]) or static written templates, the experimental design forces the Large Language Model to manipulate variables across phases. In single-animal scenes, the system learns each perceptual category—or, rather, every single exemplar (K1)—and its corresponding category label (K2), storing them as variables (see Figure 9). In dual-animal scenes, it retrieves these variables and dynamically operates on them, since they are inserted into the context-dependent constraint—the conditional logic (K3). Ultimately, the artificial neural network retrieves and applies the stored information (K1–K3) to resolve referential ambiguity via exclusion (K4).

These variables, injected into the written textual input, directly target and apply to the current visual context. Such context-based experience encompasses the storage of perceptual categories (K1), category–word mappings (K2), and constraints (K3) as prerequisites for inferential reasoning (K4). Thus, the paradigm tests whether artificial systems can move beyond rule abstraction and fixed templates toward context-sensitive human-like reasoning. This approach highlights that our model’s inferential power arises from the successful retrieval and integration of all previously stored capacities (K1–K3) with the current visual scene to execute meaningful, adaptive inferential reasoning—triggered by context-dependent constraints using embedded variables (see Figure 10, Figure 11, Figure 12 and Figure 13)—even across completely novel stimuli. Figure 14 illustrates the experimental framework used for word learning.

Note also that in single-animal scenes, the model stores part of its foundational knowledge (K1–K2). In dual-animal scenes, the artificial neural network must first retrieve K1 and K2 before it can store the conditional rule (K3), as part of this foundational knowledge, and subsequently retrieve and apply all three components to execute inference (K4). In Large Language Models, moving from Phase 1 (single-animal scenes) to Phase 2 (dual-animal scenes)—especially when new competing entities and pseudowords are introduced—can pose a risk of degrading or unlearning prior representations unless they are reactivated. Recall that the pedagogy of sensory chunks included two core principles. First, repetition promotes memory formation and strengthening. Second, information which cannot be retrieved cannot be used (see sensorimotor rehearsal and rehearsed information testing in Section 1.1). The model was repeatedly exposed to specific sensory configurations and engaged in repeated action practice throughout K1–K4. Testing previously rehearsed content (K1–K2) before adding novel information (K3) ensured that the system reconnected with part of its knowledge base to mitigate interference and memory decay. Thus, this retrieval process consolidated previously learned memories—while competing stimuli existed in a shared context—and was essential for preventing catastrophic forgetting.

4.4. Hyperparameters and Fine-Tuning Efficiency

To perform this fine-tuning stage efficiently without updating the entire parameter space of the BLIP-2 architecture, a Low-Rank Adaptation (LoRA) [76] was used. This technique freezes the original pre-trained weights and injects smaller, trainable rank decomposition matrices into the attention blocks, optimising memory usage without compromising performance parity.

The authors propose a LoRA-based fine-tuning approach for the last layer of the Large Language Model during Phase 2 (dual-animal scenes). The main parameters of LoRA applied in this research are in

Table 2. LoRA hyperparameters values.

LoRA Parameter Name	LoRA Parameter Value
LoRA rank (r)	32
LoRA Alpha (α)	64
LoRA Dropout	0.05

.

The LoRA rank ($r$) determines the size of the matrices. LoRA introduces an additional update to the original weights of the last layer of the Large Language Model. This term is a product of two matrices, A and B, which have lower dimensions than the original weight matrix W (see Equation (1)).

$$\Delta W={\displaystyle \frac{\alpha }{r}}BA$$

(1)

This parameter was set to 32, which allows the Large Language Model to learn complex tasks. The Alpha (α) parameter modulates the intensity of the LoRA update. In this case, α was set to 64, giving the LoRA term a scaling factor of 2, which implies a relatively high adaptation weight. Finally, dropout was set to 5% to regularise training and reduce overfitting. The training data has been uploaded to Zenodo [71], and the complete code is available from the authors upon request.

Mathematically, for a pre-trained weight matrix $W_0\in {\mathbb{R}}^{d·k},$ the weight update is decomposed via a low-rank representation (see Equation (2)).

$$h=W_{0}x +\Delta Wx =W_{0}x +BA$$

(2)

where $A\in {\mathbb{R}}^{r·k}$ and $B\in {\mathbb{R}}^{d·x}$ are the trainable matrices, and $r$ is significantly smaller than the model’s hidden dimensions $\left(r\ll \min \left(d,k\right)\right)$. This approach drastically reduces the number of trainable parameters while maintaining performance parity with full fine-tuning.

The authors employed a relatively high number of training epochs (~200) alongside a learning rate of 1 × 10⁻⁵ to guarantee stable acquisition of the initial categories. For dual-animal scenes, training was conducted with a learning rate of 1 × 10⁻⁴ for a maximum of 18 epochs. In both experimental phases, an Early Stopping regularisation criterion was continuously monitored, halting the optimisation process immediately once validation loss plateaued to prevent overfitting.

While extended fine-tuning on a narrow, synthetic dataset can indeed lead to overfitting or the overwriting of generalised pre-trained representations in full-parameter training, the use of Low-Rank Adaptation (LoRA) inherently mitigates this risk and acts as a structural defence against catastrophic forgetting. By keeping the original base weights frozen and restricting updates to low-rank matrices (r = 32, α = 64, with 5% dropout), the model preserves its core pre-trained visual knowledge while adapting specifically to the novel category-word mappings of the Fictitious-Animal Paradigm without corrupting foundational representations.

To increase the replicability of this research,

Table 3. Training hyperparameter values.

Hyperparameter Name	Familiar Creature	Novel Creature
Maximum Epochs	200	18
Stop Epoch Number in Training	~185	~12
Learning Rate	1 × 10−5	1 × 10−4
Early Stopping Patience	5	5

details the training parameters applied across the single-animal and dual-animal experimental phases. An Early Stopping patience of 5 epochs was implemented—halting training if validation loss increased across five consecutive epochs—to prevent overfitting. In the single-animal phase, Early Stopping triggered the conclusion of training at 185 epochs, whereas in the dual-animal phase, it halted training at 12 epochs.

5. Results

To evaluate whether the fine-tuned BLIP-2 model successfully developed the four computational capacities, we analysed 64 evaluation annotations for inference. The 32 dual-animal scenes involving two queries for a familiar fictitious animal and two queries for a novel creature produced these evaluations. Consequently, we obtained disaggregated inference results for both the exclusion of the familiar category and the mapping of the novel creature.

Each trial required the system to retrieve previously stored knowledge (K1–K3), resolve real-time referential ambiguity by applying the context-dependent constraint—prohibiting the assignment of a novel label Y to a familiar creature X—and accurately identify the queried creature’s spatial position. Performance was evaluated using Exact Match (EM) for spatial localisation alongside BLEU and BERTScore to measure syntactic and semantic fidelity for each item [76,77,78] relative to the continuous workflow as a fully integrated sequence of computational capacities.

To assess performance granularly across category types,

Table 4. Disaggregated inference results.

Disaggregated Inference Results	Familiar Creature	Novel Creature
Exact Match	0.8125	0.9688
BLEU Score	0.9688	0.9948
BERTScore	0.9682	0.9685

reports the disaggregated inference results for familiar and novel creatures. The model achieves strong performance across all metrics, with Exact Match reaching 0.8125 for familiar creatures and 0.9688 for novel creatures. Furthermore, BLEU scores record 0.9688 and 0.9948, while BERTScore remains exceptionally stable at 0.9682 and 0.9685, respectively. These metrics demonstrate that the model successfully generalises the acquired mappings without performance degradation between familiar and newly introduced categories (see Table 4).

During intensive fine-tuning on newly introduced categories, which employed fewer images but a higher epoch density, a temporary recency bias emerged. This bias favoured the newly acquired vocabulary. However, Exact Match enforced strict character-by-character lexical alignment. Consequently, minor variations in how the model decoded target labels for familiar items lowered this strict metric, despite semantic correctness. This was confirmed by the exceptionally high BLEU score of 0.9688. Conversely, novel creatures benefited from recent, highly guided training protocols that prioritised precise matching, whereas familiar items may have experienced minor interference or decoding drift regarding the exact expected string.

Although the repetition and rehearsed information testing described in Section 4.3 successfully prevented structural unlearning, this slight discrepancy in Exact Match reflects a combination of residual interference and a standard recency effect inherent to sequential fine-tuning. When presented with a familiar target query (Figure 11 and Figure 12), the model successfully retrieved the stable visual definition stored in memory and mapped it onto the correct spatial coordinate. Crucially, when confronted with a novel target query (Figure 13 and Figure 14), the system applied the Mutual Exclusivity rule to map the novel label (gint) to the unlabelled category through active visual inference.

The description of the familiar target remains highly stable due to mass repetition training. In contrast, the description of the novel category was generated dynamically during the real-time, context-based inference process. Although novel target instances (Figure 12 and Figure 13) exhibit minor morphological variations across trials—yielding unstable visual descriptions—the model successfully recognises and localises the target entity. This indicates that despite possessing diverse visual features, these instances share core invariant traits. While these overlapping features increase task complexity, the model achieves a remarkable accuracy. This high performance is significant. It demonstrates that the model is not merely retrieving a memorised response, but is performing inference based on less consolidated perceptual representations. This further supports the view that the Large Language Model operates adaptively by integrating capacities K1–K4.

Furthermore, the authors include empirical visualisations of the training and validation loss curves to prove smooth convergence, and cross-attention heatmaps to demonstrate that the model is actively focusing on the correct label assignment and spatial location during inference.

In the graphical representation of Phase 1 (single-animal scenes), both loss curves start around 10, but decrease exponentially to values below 0.5 within approximately 1000 iterations. The validation loss steadily declines and remains stable through the end of training, which confirms that the model learns and generalises effectively as training progresses. When validation loss failed to decrease for five consecutive iterations, Early Stopping was triggered to prevent overfitting. In contrast, the training loss shows irregular fluctuations after about 500 iterations, as mini-batches begin to include more difficult learning instances. However, this trend does not indicate a training failure, as the overall trajectory continues to decline steadily.

In Phase 2 (dual-animal scenes), both loss curves initiate around 2 and drop rapidly in an exponential fashion. Beyond approximately 300 iterations, both curves drop below 0.1 and converge tightly, reflecting comparable performance across both training and validation sets. As in the preceding phase, Early Stopping halts training if validation loss plateaus for five consecutive iterations (see Figure 15 and Figure 16).

6. Conclusions

This study demonstrates that feeding Large Language Models with structured cross-domain input successfully drives the storage of perceptual categories (here, single exemplars), category-word mappings, and conditional IF–THEN rules such as the Mutual Exclusivity rule. This input comprises complementary non-linguistic and linguistic information derived from visual contexts and language (written text). The stored data is then retrieved to execute human-like inferential reasoning grounded in current visual contexts.

Unlike models relying on decontextualised, meaningless theoretical abstractions (see Anderson [50]; Chomsky [69]), our proposal shifts the focus toward a framework in which structured input facilitates adaptive, context-based word learning. This new framework mirrors how biological language acquisition entails storing, retrieving, and applying complementary non-linguistic and linguistic information to execute meaningful, real-time inferential reasoning in current visual environments. Note also that in naturalistic and enterprise environments, language is never acquired or processed through decontextualised, text-only abstractions; rather, it relies fundamentally on cross-domain, multimodal anchoring. A text-only baseline lacks real utility or practical sense—both in natural human cognition and in robust enterprise applications—as it would subvert the very purpose of the proposed architecture.

The authors advanced a novel theoretical contribution, entitled “The Architecture of the Developmental Milestones in Human Language Acquisition: From Vocabulary to Linguistic Motor Programmes”. Inspired primarily by neuroscience, the proposed framework bridged biological development and artificial learning. More precisely, our theoretical contribution comprised four developmental milestones (Ms) reflected as four cognitive capacities in humans and operationalised and tested as four computational capacities (Ks) in Large Language Models for word learning: (K1) storing perceptual categories, (K2) storing category–word mappings, (K3) storing the Mutual Exclusivity rule, and (K4) deploying exclusion-based inferential reasoning.

Crucially, the Mutual Exclusivity rule was not verbalised as a theoretical, abstract rule for storage, but as a context-dependent constraint—“I must not assign the novel label Y to the familiar creature X”—which was instantiated, for example, as ‘I must not assign the novel label gint to the familiar creature flug’. The authors suggest that this restriction or prohibition constituted a conditional IF-THEN rule (‘IF C₁ == W₁ → THEN C₁ != W₂’) that triggered inferential reasoning. The results indicated that the model’s inferential performance was highly consistent with the conditional IF–THEN rules formalised in this framework as follows:

○

For the exclusion of the familiar visual referent, IF C₁ == flug → THEN exclude C₁

○

For the resolution of the inference, IF C₁ == flug → THEN C₁ != gint, THEREFORE C₂ == gint.

The fact that the model successfully executed these operations confirmed that referential ambiguity resolution (K4) relied on the successful prior storage of K1–K3. This demonstrated that the artificial neural network stored and flexibly operated on formal variables (e.g., label X and category Y) to achieve real-time, context-based inference.

Moreover, within this architecture, the biological process of Long-Term Potentiation served as a functional analogy for the mathematical optimisation of synaptic weights. As established by Hebbian principles, mass repetition progressively strengthens synaptic connections. In our model, this repetition transformed learning from a weak association to a robust representation of perceptual categories, their corresponding mappings, and a context-dependent constraint. This change provided the stability the system needs to deploy conditional logic during real-time inference.

Future research will focus on three main areas. First, subsequent analyses will further explore word learning through ambiguous sentences such as “The nat is looking at the pog with the telescope” (see Figure 6). Second, further assessments will determine whether injecting conditional IF–THEN rules as context-dependent constraints into the Large Language Model input enhances the acquisition of both grammar and concrete syntactic structures through deliberative inferential reasoning:

○

IF a novel animal is introduced, THEN use the frame “This is a X”

○

IF this animal is presented alongside an identical counterpart, THEN use the frame “These are NUMBER(two) X + s”

Third, the authors will investigate both syntactic proficiency and rule abstraction through sensory chunking (see Mendoza & Zulueta [33] in Section 1.1), inserting this segmentation as Large Language Model input. By altering the system’s fundamental unit of computation, sensory chunking mitigates cognitive load during information processing, presumably facilitating acquisition and enabling the automaticity required for planning and producing action sequences in humans (see chunking in Miller [79] and motor chunking and automaticity in Graybiel [3], Graybiel & Grafton [36], and Smith & Graybiel [37]). The authors intend to translate this approach into Large Language Models to investigate whether it yields parallel computational efficiencies, comparing system performance with and without chunk-based sentences that use constructional frames such as “[This is] [a X]” and “[These are] [NUMBER(two) X + s]”. Methodologically, the authors will utilise an artificial language because, although models such as BLIP-2 already possess the pluralisation rules of English, our goal is to ensure the model acquires these rules de novo without relying on any prior knowledge of natural language. Future studies will also include multi-exemplar variations (such as colour and morphology) and expand the evaluation dataset to hundreds of unique configurations, reporting the exact Clopper-Pearson confidence intervals.

In summary, this study presented an innovative theoretical framework that conceptualised core developmental milestones in human language acquisition as four cognitive capacities. These capacities inspired four computational capacities (K1–K4) in Large Language Models for word learning. In doing so, we established a rigorous methodology to evaluate how structured cross-domain input (complementary non-linguistic and linguistic stimuli) enables the acquisition of perceptual categories, category-word mappings, and conditional IF–THEN rules—specifically the Mutual Exclusivity rule—to execute adaptive, context-based, meaningful inferential reasoning akin to human word learning.

7. Patents

The pedagogy of sensory chunks mentioned in this research is copyrighted as NA-0395/18 (year, 2018) and NA-0354/19 (year, 2019) under the name of Veronica Mendoza Fernandez.