From Prompting to Multi-Agent AI: Controlled Generation of CEFR-Specific Language Learning Content

Featured Application

The proposed multi-agent AI approach can be applied to digital language-learning platforms to generate level-specific learning content using explicitly configured CEFR-related requirements.

Abstract

Generative artificial intelligence is increasingly being used to support content creation in digital language-learning environments; however, generating educational materials that consistently reflect different proficiency levels remains a challenge. This study investigates an orchestrated multi-agent AI architecture for generating language-learning content targeting different CEFR proficiency levels. The proposed architecture employs an orchestrator that routes content-generation requests to independently configured, level-specialized agents operating under predefined lexical, structural, and instructional requirements, without inter-agent collaboration or deliberation. To distinguish the effects of explicit level-specific constraints from those associated with agent-based orchestration, a controlled experiment compared three generation conditions using the same underlying large language model: conventional prompt-based (PB) generation, constrained prompting (CP), and multi-agent (MA) generation. CP incorporated the same core level-specific requirements and vocabulary resources as MA without employing agent-based orchestration. The three conditions were evaluated using 100 matched tasks across five intended CEFR levels (A1–C1), resulting in 300 generated stories. The outputs were assessed in terms of text length, lexical diversity, syntactic characteristics, and adherence to predefined generation constraints. Within MA, all six continuous linguistic measures showed significant differences across intended proficiency levels, although adjacent-level differentiation varied across linguistic dimensions. Across the three conditions, PB differed from CP and MA on several linguistic measures, whereas CP and MA exhibited greater similarity, with statistically significant differences in 12 of 30 planned comparisons. Both CP and MA achieved 100% adherence to the evaluated story-length, target-vocabulary-count, and comprehension-question-count requirements in the analyzed outputs. Overall, the findings suggest that explicit constraints and vocabulary resources contribute substantially to controlled proficiency-targeted generation, while the orchestrated multi-agent architecture provides modular organization and is associated with selected level-dependent linguistic differences rather than consistent superiority over CP. The study therefore focuses on controlled linguistic differentiation and operational constraint adherence, rather than independent validation of CEFR proficiency levels.

1. Introduction

Generative artificial intelligence (GenAI) has rapidly expanded the possibilities for creating educational content in digital learning environments. Large language models (LLMs) can support a wide range of language-learning and teaching activities, and recent research has increasingly examined their use across different language skills, instructional strategies, and educational contexts [1,2]. In language education, these capabilities are particularly attractive because learning materials frequently need to vary according to learners’ proficiency levels, vocabulary knowledge, linguistic complexity, and instructional objectives. The ability to generate such materials on demand may therefore support the development of more flexible and scalable digital language-learning environments. Recent reviews nevertheless emphasize the need for more systematic empirical evaluation of GenAI applications in language education [1,2].

Despite these opportunities, the use of general-purpose LLMs for educational content generation presents an important challenge: prompting a model to produce content for a particular proficiency level does not necessarily ensure systematic control over the linguistic and instructional characteristics of the generated material. Recent research on CEFR-controlled generation similarly identifies reliable level control as a non-trivial problem, particularly when multiple linguistic characteristics must be balanced simultaneously [3]. Language-learning content intended for different proficiency levels may vary across lexical, syntactic, structural, and task-related characteristics. Consequently, specifying a proficiency label such as A1, B1, or C1 within a prompt may be insufficient when multiple level-specific requirements must be satisfied simultaneously and consistently.

The Common European Framework of Reference for Languages (CEFR) provides a widely used framework for describing language proficiency through common reference levels and associated descriptors [4]. The framework provides a basis for language curricula, teaching and learning materials, and the assessment of language proficiency [4]. From the perspective of AI-generated learning materials, CEFR-oriented content generation can therefore be viewed as a multidimensional control problem rather than a simple text-generation task. Recent work on automated CEFR-oriented material generation has operationalized this challenge through combinations of lexical complexity, syntactic difficulty, and other level-sensitive features [3]. Designing generative systems capable of maintaining systematic distinctions across intended proficiency levels therefore represents an important opportunity for AI-supported language-learning content development.

Recent developments in agentic AI offer an alternative to relying on a single general-purpose prompting process. LLM-based multi-agent systems provide a framework in which complex tasks can be distributed across specialized agents with differentiated roles, capabilities, and interaction mechanisms [5,6,7]. Recent studies have extended multi-agent approaches to educational contexts, including intelligent tutoring and collaborative learning activities [8,9]. Instead of requiring a single model interaction to satisfy multiple requirements simultaneously, specialized agents can be assigned explicit roles, resources, and constraints. Such specialization may be particularly suitable for language-learning content generation, where proficiency-specific requirements can be explicitly represented and managed. However, specialization can also be implemented through detailed single-prompt configurations without a multi-agent architecture. Therefore, the contribution of explicit level-specific prompting and resources must be distinguished from any additional contribution associated with agent-based orchestration. In this study, the term multi-agent refers specifically to an orchestrated specialization architecture in which a centralized orchestrator routes tasks to independently configured, proficiency-specific agents. Unlike collaborative or deliberative multi-agent systems, the proposed architecture does not involve inter-agent negotiation, critique, or joint content generation.

Motivated by these challenges, this study investigates an orchestrated multi-agent AI approach for generating CEFR-oriented language-learning content and compares it with two prompting-based alternatives. The proposed architecture uses an orchestration mechanism to route generation requests to level-specific agents that encapsulate predefined linguistic, structural, and instructional configurations. To distinguish architectural organization from the effects of detailed prompting, the experimental design includes three conditions: conventional prompt-based generation (PB), constrained prompting (CP), and multi-agent generation (MA). PB uses a general proficiency-level prompt, whereas CP incorporates the same core level-specific requirements and vocabulary resources as MA through a single constrained prompt, without employing agent-based orchestration. This design allows the contributions of explicit constraints and vocabulary resources to be examined alongside the additional differences associated with the multi-agent implementation, while recognizing that differences in instructional detail may also influence the CP–MA comparison. Against this background, the study addresses the following research questions:

RQ1. 

To what extent does the multi-agent AI approach produce systematic linguistic differentiation across intended CEFR proficiency levels?

RQ2. 

How do conventional prompt-based generation, constrained prompting, and multi-agent generation compare in terms of the linguistic characteristics of generated language-learning content and adherence to predefined operational constraints?

Accordingly, this study makes three main contributions. First, it proposes an orchestrated architecture in which proficiency-specific generation logic, resources, and requirements are encapsulated within level-specialized agents and selected through a centralized orchestration mechanism. Second, it introduces a controlled three-condition experimental design that distinguishes general CEFR prompting (PB), explicit level-specific constrained prompting (CP), and multi-agent orchestration (MA), thereby supporting a more controlled examination of the differences associated with explicit prompting constraints, vocabulary resources, and orchestrated agent specialization. Third, it provides a matched-task empirical evaluation across intended CEFR levels using multiple linguistic measures and objectively verifiable operational constraints. Rather than treating linguistic differentiation or constraint adherence as independent evidence of CEFR validity, the study evaluates the extent to which the three generation approaches produce controlled and measurable differences across intended proficiency levels.

The remainder of this paper is organized as follows. Section 2 describes the materials and methods, including the research design, multi-agent architecture, level-specific agent configuration, content-generation procedure, experimental setup, and evaluation methodology. Section 3 presents the experimental results. Section 4 discusses the findings, conceptual and methodological contributions, practical implications, limitations, and directions for future research. Finally, Section 5 concludes the paper by summarizing the main findings and contributions.

2. Materials and Methods

2.1. Research Design

This study adopts a comparative experimental design to investigate proficiency-targeted language-learning content generation using three generative AI conditions: conventional prompt-based (PB) generation, constrained prompting (CP), and the proposed multi-agent (MA) approach. The design addresses two complementary aspects of the generation process: whether the MA approach produces systematic linguistic differentiation across intended CEFR proficiency levels and how the three generation conditions differ in terms of linguistic characteristics and adherence to predefined operational constraints.

The study covers five intended CEFR proficiency levels: A1, A2, B1, B2, and C1. In the MA approach, these levels are represented by specialized content-generation agents with differentiated requirements concerning content length, target vocabulary, linguistic structures, narrative characteristics, and comprehension activities. The CP condition incorporates the same core level-specific requirements and vocabulary resources as MA but applies them through direct single-prompt generation without an orchestration mechanism or level-specific agent routing. The PB condition, by contrast, uses a general CEFR-level instruction without these detailed operational requirements. This three-condition design supports a more controlled comparison of explicit prompting constraints, vocabulary resources, and orchestrated agent specialization. However, differences in the level of instructional detail between CP prompts and MA agent configurations should be considered when interpreting CP–MA differences.

A common set of 100 content-generation tasks was applied under all three experimental conditions. The task set comprised 20 topics organized into five thematic categories, with the same 20 topics reused across A1–C1. Thus, each condition produced 100 outputs, comprising 20 outputs at each intended CEFR level, resulting in a total dataset of 300 generated stories. In the PB and CP conditions, the tasks were performed through direct interaction with the same underlying generative model, whereas in the MA condition, the tasks were processed through the orchestration mechanism and delegated to the corresponding proficiency-specific agent. Using identical level–topic combinations across the three conditions provided a matched experimental structure while limiting variation attributable to topic selection.

The individual generated story constituted the primary unit of analysis. Outputs were matched by Task_ID, intended CEFR level, and topic across PB, CP, and MA, enabling within-task comparisons among the three generation conditions.

The generated outputs were evaluated using linguistic measures of text length, lexical diversity, and syntactic characteristics. In addition, CP and MA outputs were evaluated for adherence to the operational requirements explicitly provided to both conditions, including story-length range, target-vocabulary count, and comprehension-question count. Because the PB condition was not supplied with these same explicit constraints, its outputs were not interpreted as a directly comparable test of constraint adherence. The evaluation focuses on measurable linguistic differentiation and adherence to author-defined operational requirements rather than independent validation of CEFR proficiency appropriateness. The subsequent subsections describe the multi-agent architecture, level-specific configurations, content-generation procedure, experimental setup, evaluation metrics, and statistical analysis.

2.2. Multi-Agent System Architecture

The proposed system adopts an orchestrated architecture in which content-generation requests are centrally coordinated and delegated to specialized agents according to the requested language proficiency level (Figure 1). The architecture separates request interpretation and task routing from level-specific content generation, enabling proficiency-related generation logic, resources, and constraints to be encapsulated within dedicated agents under a common coordination framework.

Figure 1. Proposed multi-agent system architecture for CEFR-oriented language-learning content generation.

At the core of the architecture is an orchestrator agent that serves as the entry point for content-generation requests. The orchestrator interprets three primary elements: the target proficiency level, the number of stories to be generated, and the requested theme. Based on the identified proficiency level, the request is delegated to the corresponding specialized story-generation agent. The orchestrator therefore functions primarily as a coordination and routing layer rather than directly generating educational content.

The content-generation layer consists of five specialized agents corresponding to the A1, A2, B1, B2, and C1 levels. Each agent encapsulates its own generation instructions, content parameters, vocabulary resources, and level-specific constraints. This specialization allows proficiency-dependent generation logic to be represented within separately managed components rather than requiring a single general-purpose configuration to accommodate the full range of proficiency levels.

The architecture is characterized as an orchestrated multi-agent system based on functional specialization and centralized task routing, rather than inter-agent collaboration or deliberation. This distinction is consistent with the broader multi-agent literature, which encompasses different forms of agent organization and coordination [5,6,7]. For an individual generation task, the orchestrator selects one proficiency-specific agent, and the specialized agents do not communicate, negotiate, critique, or jointly generate the output. Thus, the architectural distinction lies in the modular separation and routing of level-specific generation components rather than in collaborative problem solving. Although CP and MA share the same core level-specific requirements and vocabulary resources, differences in the granularity of their implementation instructions may also influence the observed CP-MA results. Consequently, these differences should not be attributed exclusively to architectural organization.

Agent specialization is further supported by level-specific lexical resources. Before story generation, the selected agent accesses the vocabulary resource associated with its proficiency level to support target-vocabulary selection in accordance with its predefined configuration.

Following generation, the resulting story and associated learning-content elements are returned in a structured format. As illustrated in Figure 1, the overall architecture integrates request orchestration, proficiency-specific agent selection, lexical-resource utilization, and structured language-learning content generation. The detailed configurations of the five specialized agents are presented in Section 2.3.

2.3. Level-Specific Generation Configuration

The proposed multi-agent framework differentiates content generation across five intended CEFR proficiency levels: A1, A2, B1, B2, and C1. Each specialized agent operationalizes its target level through a predefined configuration covering story length, target vocabulary, linguistic structures, narrative characteristics, and comprehension activities. These configurations should not be interpreted as direct CEFR prescriptions. Rather, they combine proficiency-oriented linguistic expectations with author-defined operational constraints introduced to provide consistent and measurable generation targets across levels.

As summarized in Table 1, the configurations introduce progressively differentiated generation requirements across the five intended proficiency levels. Story-length ranges, target-vocabulary quotas, and comprehension-question counts were defined as operational design parameters for the generation framework and experimental evaluation; they are not prescribed by the CEFR itself. The grammatical structures listed under linguistic focus were also selected by the authors as operational indicators of increasing linguistic complexity across proficiency levels, rather than as mandatory CEFR grammar specifications. Their selection was informed by the general progression of communicative and linguistic competence described in the CEFR [4], but the assignment of individual grammatical structures to specific levels represents an author-defined design decision. The linguistic focus was differentiated from relatively basic grammatical structures at the lower levels toward more complex grammatical and idiomatic structures at the higher levels, while comprehension activities incorporated inference- and critical-thinking-oriented requirements at the higher levels. Accordingly, Table 1 represents the operationalization adopted in this study rather than a direct mapping of formal CEFR descriptors to fixed numerical thresholds. All configurations were fixed prior to generation and remained unchanged throughout the experiment.

Table 1. Level-specific generation configurations used in the proposed framework.

CEFR Level	Story Length *	Target Vocabulary *	Comprehension Questions *	Linguistic/ Narrative Focus
A1	80–120 words	6–8	3	Present simple; there is/are; basic questions
A2	130–180 words	8–10	4	Present continuous; past simple; can/can’t; comparatives
B1	200–280 words	10–12	5 (≥1 inference)	Present perfect; past continuous; first conditional; modals
B2	300–420 words	12–15	5 (1–2 inference/opinion)	Second/third conditional; passive voice; reported speech
C1	450–650 words	15–18	6 (critical thinking)	Mixed conditionals; inversion; idiomatic language

* Story-length ranges, target-vocabulary quotas, and comprehension-question requirements are author-defined operational constraints used for controlled generation and evaluation; they are not fixed thresholds prescribed by the CEFR.

The CEFR provides the general proficiency framework [4]. The specific grammatical structures, numerical thresholds, and comprehension-question requirements represent author-defined operationalizations intended to support measurable differentiation across levels.

Each specialized agent was additionally associated with a level-specific vocabulary resource stored in JSON format, with each lexical entry explicitly labeled according to its intended CEFR level. The resources contained 909 entries for A1, 962 for A2, 1119 for B1, 707 for B2, and 127 for C1. The vocabulary resources were developed using Oxford vocabulary lists as their source and organized into level-specific JSON inventories. Each JSON entry contained a vocabulary item, its intended CEFR level, an English definition, a Turkish meaning, and example sentences. The CEFR labels associated with these entries were used as operational classifications for vocabulary selection and were not independently validated as part of the present study. During generation, the corresponding resource was used to support the selection of topic-relevant target vocabulary items within the predefined numerical range for each level. These resources were fixed throughout the experiment and were used in both the CP and MA conditions. They functioned as operational generation resources and are not presented as exhaustive or normative CEFR vocabulary inventories. The smaller C1 resource may have restricted the range of available target vocabulary and should therefore be considered when interpreting lexical characteristics at this level.

Across all proficiency levels, the agents additionally followed a common set of narrative requirements: third-person narration, a single main character and scenario, adherence to the specified topic and theme, and a neutral everyday-language tone. These requirements were held constant across levels, while the level-specific parameters presented in Table 1 provided the operational basis for differentiating the generated content. The CP condition incorporated the same core level-specific requirements and vocabulary resources through direct prompting, although its prompt instructions were not identical in detail to the MA agent configurations.

2.4. Content Generation Procedure

The content-generation procedure was implemented separately for the three experimental conditions while preserving the same underlying generation tasks and target proficiency levels. In all conditions, each task specified the intended CEFR level and topic, and each generation was conducted as an independent model interaction without conversational history from previous tasks.

In the PB condition, the underlying generative model received a general prompt specifying the intended CEFR level and topic and requesting a language-learning story followed by comprehension questions. No explicit story-length range, target-vocabulary quota, level-specific vocabulary resource, grammatical configuration, or fixed question count was provided.

In the CP condition, the same underlying model received a fixed level-specific prompt containing the same story-length range, target-vocabulary requirement, linguistic focus, comprehension-question requirement, and common narrative requirements used in the corresponding MA configuration. The same level-specific vocabulary resource available to MA was also provided for target-vocabulary selection. However, generation occurred directly through a single prompt without orchestrator-based routing or delegation to a specialized agent.

In the MA condition, each request was processed by the orchestrator and delegated to the corresponding level-specific agent. Once activated, the selected agent accessed the vocabulary resource associated with its proficiency level and selected target vocabulary in accordance with its predefined configuration. Story generation then proceeded according to the level-specific requirements defined in Section 2.3, with the selected vocabulary incorporated into the generated content. Comprehension questions were generated as part of the same process.

Although CP and MA shared the same core operational requirements and level-specific vocabulary resources, their instructional configurations differed in how these requirements were implemented. In CP, story-length ranges, target-vocabulary requirements, linguistic focus, and comprehension-question requirements were specified within a single consolidated prompt. In MA, requests were routed to level-specific agents, which accessed the corresponding vocabulary resources and applied additional guidance concerning grammatical structures and vocabulary selection. Thus, while both conditions followed the same core generation requirements, MA incorporated agent-specific instructional guidance and orchestrator-based task delegation that were absent from CP. Consequently, the observed CP-MA differences cannot be attributed exclusively to architectural organization.

Across the three conditions, the generated story constituted the primary textual output used for linguistic analysis, while the associated comprehension questions were retained separately for evaluation of the relevant operational requirement.

The experimental workflow was designed to execute each level–topic task once under each condition, without an automatic regeneration or retry mechanism. The resulting dataset comprised 300 analyzed outputs, with 100 outputs per condition. Because complete intermediate execution logs were not retained, the study does not independently verify first-pass generation reliability. Accordingly, reported constraint-adherence rates refer to the final analyzed outputs and should not be interpreted as evidence of first-attempt success rates.

2.5. Experimental Setup and Baselines

To evaluate the proposed multi-agent framework, a controlled comparative experiment was conducted using three generation conditions: conventional prompt-based generation (PB), constrained prompting (CP), and multi-agent generation (MA). The same underlying generative model, intended CEFR levels, thematic inputs, and generation tasks were used across all three conditions. The inclusion of CP was intended to provide a stronger prompting-based comparator, allowing differences associated with explicit level-specific requirements and vocabulary resources to be examined alongside those associated with orchestrated agent specialization.

The experiment covered five intended proficiency levels (A1–C1), with 20 generation tasks defined for each level. The task set comprised 20 topics organized into five thematic categories: daily life and routines, education and learning, travel and cultural experiences, technology and digital life, and social interaction and problem-solving. The same 20 topics were reused across all five intended proficiency levels, resulting in 100 matched level–topic tasks per condition. The same 100 tasks were executed under PB, CP, and MA, yielding a total dataset of 300 generated stories (100 PB, 100 CP, and 100 MA outputs). This matched design enabled within-task comparisons across the three conditions while reducing variation attributable to topic selection.

In the PB condition, each task was submitted directly to the underlying generative model using a single standardized prompt specifying the requested CEFR level, topic, and language-learning purpose. The same prompt template was used for all 100 PB tasks, with only the CEFR level and task/topic fields changed: “Generate an English language-learning story appropriate for CEFR [LEVEL] learners on the topic ‘[TASK/TOPIC]’. The story should be suitable for language-learning purposes. After the story, generate comprehension questions appropriate for the same CEFR level.”

PB was designed to represent general CEFR-oriented prompting. Accordingly, the prompt did not specify story length, target-vocabulary requirements, particular linguistic structures, or a fixed number of comprehension questions, nor did it use level-specific lexical resources, orchestration, or specialized agents.

In the CP condition, each task was submitted directly to the same underlying generative model using a fixed prompt template corresponding to the intended proficiency level. The CP prompts incorporated the same core level-specific story-length ranges, target-vocabulary requirements, linguistic focus, comprehension-question requirements, common narrative requirements, and vocabulary resources used in the corresponding MA configurations. Within each proficiency level, the CP prompt structure remained fixed across tasks, with the topic field changed according to the matched task. Unlike MA, CP used direct single-prompt generation without orchestrator-based routing or delegation to a level-specific agent. Although the core operational requirements and vocabulary resources were shared, the CP prompts and MA agent configurations were not identical in instructional detail. CP therefore served as a stronger prompting comparator rather than an architecturally isolated control condition.

For reproducibility, the following A1 constrained prompt illustrates the CP implementation: “Generate an English language-learning story appropriate for CEFR A1 learners on the topic ‘[TOPIC]’. The story must be 80–120 words long. Select and incorporate 6–8 target vocabulary items from the provided A1 vocabulary resource. Use present simple, there is/are, and basic questions. Use third-person narration with one main character and one scenario, adhere to the specified topic/theme, and maintain a neutral everyday-language tone. After the story, generate exactly 3 comprehension questions. Return the story, selected target vocabulary, word count, and comprehension questions in a structured format.”

The remaining CP prompts followed the corresponding level-specific parameters summarized in Table 1.

In the MA condition, each matched task was processed through the proposed system and routed by the orchestrator to the corresponding proficiency-specific agent. The selected agent applied the predefined level-specific configuration and vocabulary resource described in Section 2.2, Section 2.3 and Section 2.4.

All three conditions used Claude Sonnet 5 (claude-sonnet-5) within Claude Code using Claude Agent SDK sessions. Model-related generation parameters were kept consistent across conditions where technically applicable. Temperature, top-p, and maximum output-token settings were not separately parameterized and therefore remained at the system defaults. Each generation task was executed independently without conversational history from previous tasks. One final generated output per task and condition was included in the analysis. The available experimental records contain the final outputs used in the study but do not provide sufficiently detailed generation logs to retrospectively verify the complete execution history of every task. Accordingly, the study does not interpret the analyzed outputs as evidence of first-pass generation reliability. Table 2 summarizes the experimental conditions.

Table 2. Experimental design for PB, CP, and MA content generation.

Experimental Characteristic	Prompt-Based Generation (PB)	Constrained Prompting (CP)	Multi-Agent Generation (MA)
Underlying LLM	Claude Sonnet 5	Claude Sonnet 5	Claude Sonnet 5
Intended CEFR levels	A1, A2, B1, B2, C1	A1, A2, B1, B2, C1	A1, A2, B1, B2, C1
Tasks per level	20	20	20
Total outputs	100	100	100
Themes/tasks	Identical matched task set	Identical matched task set	Identical matched task set
Generation mechanism	Direct single-prompt generation	Direct constrained single-prompt generation	Orchestrator + level-specific agent
Level-specific operational requirements	Not explicitly specified	Explicitly specified	Encoded in level-specific agent configuration
Level-specific vocabulary resources	Not provided	Same resources as MA	Accessed by level-specific agent
Story-length range	Not specified	Same as MA	Level-specific
Target-vocabulary requirement	Not specified	Same as MA	Level-specific
Comprehension-question count	Not specified	Level-specific	Level-specific
Instructional detail	General CEFR-oriented instructions	Level-specific constrained prompt	Level-specific agent configuration

The matched experimental design supported two complementary analyses. First, the 100 MA outputs were examined across A1–C1 to determine whether systematic variation in linguistic characteristics was observable across intended proficiency levels. Second, the matched PB, CP, and MA outputs were compared within each intended CEFR level and task to examine differences among the three generation conditions. Within this three-condition comparison, PB–CP differences were used to assess differences associated with introducing explicit level-specific constraints and resources, whereas CP–MA differences were examined to identify additional differences associated with the multi-agent implementation. Because CP and MA differed in both architectural organization and some aspects of instructional detail, the observed CP–MA differences cannot be attributed exclusively to orchestration. PB–MA comparisons were retained to represent the overall difference between conventional prompting and the proposed multi-agent implementation.

2.6. Evaluation Metrics

The generated outputs were evaluated across four complementary dimensions: lexical characteristics, syntactic characteristics, operational constraint adherence, and comprehension activities. All lexical and syntactic measures were calculated from the generated story text only; comprehension questions were excluded from these linguistic analyses.

Lexical characteristics were evaluated using word count and the Moving-Average Type–Token Ratio (MATTR). Word count was used to characterize differences in text length across intended proficiency levels and generation conditions. Because story-length ranges were explicitly specified in CP and MA, word count was considered a partially engineered outcome rather than an entirely unconstrained indicator of linguistic differentiation. Lexical diversity was quantified primarily using MATTR with a 50-token window (MATTR-50), which reduces sensitivity to differences in overall text length [10,11]. Because the shortest generated texts were relatively close to the primary window size, MATTR was additionally recalculated using a 25-token window (MATTR-25) as a sensitivity analysis. Word counts and MATTR values were calculated in Python 3.12.7 using a consistent tokenization procedure across all generated stories. Text was converted to lowercase, and word tokens were extracted using the following regular expression, which retains contractions containing an internal apostrophe as single tokens while excluding punctuation from token counts:

[a-z]+(?:‘[a-z]+)?

Syntactic characteristics were evaluated using mean sentence length (MSL) together with three established measures of syntactic complexity: mean length of T-unit (MLT), dependent clauses per T-unit (DC/T), and complex nominals per clause (CN/C). These measures were selected to represent complementary dimensions of syntactic complexity, with MLT reflecting global complexity, DC/T reflecting clausal or subordination complexity, and CN/C reflecting phrasal complexity [12,13,14]. MSL was calculated using sentence boundaries identified with spaCy and the same word-tokenization procedure used for text-length analysis. MLT, DC/T, and CN/C were obtained using the L2 Syntactic Complexity Analyzer (L2SCA) [12]. Although L2SCA was originally developed for the analysis of human L2 writing, the present study uses these indices as comparative structural descriptors of generated texts rather than as validated measures of AI-generated-text quality or CEFR appropriateness.

Operational constraint adherence was evaluated for the CP and MA conditions, which received the same explicit level-specific requirements. Three objectively verifiable constraints were assessed: story-length range, target-vocabulary count, and comprehension-question count. Each requirement was evaluated at the individual-output level, and adherence rates were summarized by intended CEFR level and overall. Adherence rates were calculated from the final analyzed outputs and do not represent independently verified first-attempt success rates. The PB condition was not evaluated as a directly comparable constraint-adherence condition because these explicit operational requirements were not included in the PB prompts. Accordingly, adherence results were interpreted as evidence of compliance with programmed generation requirements rather than as independent evidence of CEFR appropriateness.

Comprehension activities were evaluated descriptively using the number of comprehension questions generated for each output. Question counts were examined across intended CEFR levels and generation conditions but were not treated as an independent measure of pedagogical quality or CEFR appropriateness. Because question-count requirements were explicitly specified in CP and MA, compliance with these counts was treated as an engineered constraint outcome rather than an emergent indicator of proficiency differentiation.

Collectively, these measures were used to examine systematic linguistic differentiation across A1-C1 within the MA condition (RQ1) and to compare matched PB, CP, and MA outputs within corresponding intended proficiency levels and tasks (RQ2). The evaluation therefore distinguishes compliance with explicitly programmed operational constraints from linguistic characteristics observed in the generated texts. Neither type of evidence is interpreted as independent validation of CEFR appropriateness. Table 3 summarizes the evaluation framework.

Table 3. Evaluation framework used in the study.

Evaluation Dimension	Measures
Lexical characteristics	Word count, MATTR-50, MATTR-25 sensitivity analysis
Syntactic characteristics	Mean sentence length (MSL), MLT, DC/T, CN/C
Constraint adherence	Story-length compliance, target-vocabulary-count compliance, question-count compliance (CP and MA)
Comprehension activities	Number of comprehension questions

2.7. Statistical Analysis

Descriptive statistics were first calculated for all evaluation measures according to intended CEFR level and generation condition. Continuous measures were summarized using the mean and standard deviation, together with the median and interquartile range where appropriate.

For RQ1, the 100 outputs generated under the MA condition were analyzed across the five ordered intended CEFR levels (A1–C1) to examine systematic variation in their linguistic characteristics. Overall differences across levels were evaluated using the Kruskal-Wallis test, with epsilon-squared (ε²) reported as the corresponding effect-size measure. Where overall differences were significant, adjacent levels (A1–A2, A2–B1, B1–B2, and B2–C1) were compared using Mann–Whitney U tests, with Holm adjustment applied across the four adjacent-level comparisons within each linguistic measure. Spearman’s rank correlation (ρ) between the ordered intended CEFR level and each continuous linguistic measure was additionally reported as a descriptive measure of monotonic association. Because some generation characteristics, particularly story length, were explicitly specified in the level configurations, these correlations were not interpreted as independent evidence of CEFR validity or as evidence that the architecture had autonomously learned proficiency progression. Operational constraint adherence was summarized descriptively.

For RQ2, the matched PB, CP, and MA outputs were compared separately within each intended CEFR level for each linguistic measure. Because the same generation tasks were represented under all three conditions, overall differences among PB, CP, and MA were evaluated using the Friedman test, with Kendall’s W reported as an effect-size measure. In addition, three prespecified paired contrasts were evaluated using Wilcoxon signed-rank tests: PB-CP, PB-MA, and CP-MA. Rank-biserial correlation was used as the effect-size measure for the paired contrasts. Holm’s procedure was applied across the three planned contrasts within each CEFR-level × linguistic-measure family. The PB-CP contrast was used to characterize differences associated with introducing explicit level-specific constraints and resources; the CP-MA contrast was used to examine differences between constrained prompting and orchestrated agent specialization under shared core requirements and vocabulary resources, while recognizing that differences in instructional detail may also contribute to the observed results; and PB–MA represented the overall difference between conventional prompting and the multi-agent implementation. Because these contrasts were prespecified, they were evaluated independently of whether the corresponding Friedman omnibus test reached statistical significance and are therefore referred to as planned contrasts rather than post-hoc tests.

As a robustness analysis for lexical diversity, the primary MATTR-50 analysis was repeated using a 25-token moving window (MATTR-25). The association between MATTR-50 and MATTR-25 across all 300 outputs was evaluated using Spearman’s rank correlation, and the three-condition comparisons for MATTR-25 were repeated using the same Friedman and planned Wilcoxon procedure described above.

Statistical significance was evaluated at an alpha level of 0.05, with adjusted p-values reported where Holm correction was applied. Effect sizes were considered alongside statistical significance to assess the magnitude of observed differences rather than relying exclusively on p-values.

3. Results

3.1. Descriptive Characteristics of the Generated Content

Descriptive statistics were first examined to characterize the outputs generated under the prompt-based (PB), constrained prompting (CP), and multi-agent (MA) conditions across the five intended CEFR proficiency levels. Table 4 summarizes text length, lexical diversity, sentence-level characteristics, and syntactic complexity for each proficiency level and generation condition.

Table 4. Descriptive statistics of generated content by CEFR level and generation condition.

CEFR	Condition	Word Count M (SD)	MATTR-50 M (SD)	MSL M (SD)	MLT M (SD)	DC/T M (SD)	CN/C M (SD)	Questions M (SD)
A1	MA	93.45 (5.63)	0.772 (0.052)	8.74 (0.89)	7.90 (0.83)	0.033 (0.048)	0.567 (0.312)	3.00 (0.00)
	CP	94.00 (3.31)	0.752 (0.029)	8.01 (0.70)	7.55 (0.79)	0.004 (0.017)	0.587 (0.305)	3.00 (0.00)
	PB	106.80 (41.59)	0.765 (0.051)	14.10 (2.67)	9.98 (1.48)	0.134 (0.105)	0.635 (0.346)	4.90 (1.17)
A2	MA	161.45 (4.87)	0.806 (0.028)	13.24 (1.34)	9.66 (1.26)	0.211 (0.110)	0.626 (0.156)	4.00 (0.00)
	CP	148.35 (4.07)	0.810 (0.031)	11.98 (0.58)	9.92 (0.96)	0.257 (0.097)	0.756 (0.255)	4.00 (0.00)
	PB	182.65 (15.78)	0.786 (0.030)	12.99 (1.48)	10.98 (1.25)	0.300 (0.165)	0.680 (0.245)	5.80 (0.52)
B1	MA	255.95 (10.68)	0.847 (0.020)	14.68 (1.65)	12.38 (1.84)	0.511 (0.108)	0.743 (0.215)	5.00 (0.00)
	CP	244.35 (10.36)	0.849 (0.021)	15.07 (1.34)	12.69 (1.07)	0.522 (0.136)	0.817 (0.198)	5.00 (0.00)
	PB	328.00 (20.21)	0.846 (0.023)	22.13 (3.69)	18.01 (3.65)	0.820 (0.243)	0.966 (0.178)	5.70 (0.57)
B2	MA	326.30 (25.67)	0.873 (0.021)	20.48 (1.72)	16.88 (1.81)	0.879 (0.123)	0.986 (0.182)	5.00 (0.00)
	CP	339.35 (15.35)	0.876 (0.018)	21.43 (1.99)	17.82 (2.47)	0.929 (0.186)	0.995 (0.240)	5.00 (0.00)
	PB	363.80 (10.10)	0.874 (0.017)	33.88 (7.05)	26.43 (6.32)	1.337 (0.423)	1.328 (0.274)	5.35 (0.49)
C1	MA	489.10 (32.48)	0.874 (0.014)	27.46 (3.03)	21.43 (2.75)	0.886 (0.145)	1.301 (0.327)	6.00 (0.00)
	CP	520.75 (12.49)	0.866 (0.021)	28.64 (2.79)	24.16 (3.95)	1.143 (0.283)	1.194 (0.225)	6.00 (0.00)
	PB	315.25 (22.28)	0.860 (0.016)	31.25 (6.45)	25.50 (5.03)	1.230 (0.380)	1.323 (0.224)	5.50 (0.61)

Note: Values are reported as mean (standard deviation); n = 20 per intended CEFR level and generation condition. PB = prompt-based generation; CP = constrained prompting; MA = multi-agent generation; MATTR-50 = moving-average type–token ratio with a 50-token window; MSL = mean sentence length; MLT = mean length of T-unit; DC/T = dependent clauses per T-unit; CN/C = complex nominals per clause.

Text length showed distinct descriptive patterns across the three conditions. In the MA condition, mean story length increased from 93.45 words at A1 to 161.45 at A2, 255.95 at B1, 326.30 at B2, and 489.10 at C1. CP showed a similarly ordered pattern, with mean story length increasing from 94.00 words at A1 to 148.35 at A2, 244.35 at B1, 339.35 at B2, and 520.75 at C1. PB showed a different pattern: mean story length increased from 106.80 words at A1 to 182.65 at A2, 328.00 at B1, and 363.80 at B2, followed by a decrease to 315.25 words at C1. This decrease in PB story length at C1 represents a descriptive departure from the increasing pattern observed at the preceding levels; the present analysis does not establish its underlying cause. The ordered increases observed for CP and MA should be interpreted in light of the explicit level-specific story-length ranges provided to these two conditions. Consequently, text-length progression in CP and MA represents a partially engineered outcome rather than independent evidence of proficiency differentiation.

Lexical diversity, measured using MATTR-50, varied more moderately across conditions and proficiency levels. Mean values for MA were 0.772, 0.806, 0.847, 0.873, and 0.874 from A1 through C1, respectively. The corresponding CP values were 0.752, 0.810, 0.849, 0.876, and 0.866, while PB values were 0.765, 0.786, 0.846, 0.874, and 0.860. Thus, all three conditions generally showed higher lexical-diversity values at the intermediate and upper levels than at the lower levels, although the level-wise patterns were not identical. In particular, the differences between B2 and C1 were relatively small, indicating that lexical diversity did not increase uniformly across every adjacent proficiency level.

Sentence-level and syntactic measures showed additional descriptive differences. In MA, mean sentence length increased from 8.74 words at A1 to 27.46 words at C1, while MLT increased from 7.90 to 21.43 over the same range. In CP, mean sentence length increased from 8.01 at A1 to 28.64 at C1, and MLT increased from 7.55 to 24.16. PB generally produced higher values for several syntactic measures at the intermediate and upper levels, with mean sentence length reaching 33.88 and MLT reaching 26.43 at B2 before decreasing to 31.25 and 25.50, respectively, at C1. Similar condition- and level-dependent patterns were observed for DC/T and CN/C, as reported in Table 4. These descriptive results indicate differences in the structural characteristics of generated texts but do not, by themselves, establish greater pedagogical suitability or CEFR appropriateness for any generation condition.

Comprehension-question counts were examined separately as a descriptive output characteristic. Because fixed question-count requirements were explicitly provided to CP and MA, these values primarily represent compliance with programmed operational constraints rather than independent linguistic differentiation. Both CP and MA generated three questions at A1, four at A2, five at B1 and B2, and six at C1. PB, which did not receive fixed question-count requirements, showed mean question counts of 4.90 at A1, 5.80 at A2, 5.70 at B1, 5.35 at B2, and 5.50 at C1.

3.2. Linguistic Differentiation Across Intended CEFR Levels in MA Outputs

The linguistic characteristics of the MA-generated stories were examined across the five intended CEFR proficiency levels to determine whether the outputs exhibited systematic differentiation from A1 to C1. Kruskal–Wallis tests indicated significant differences across levels for all six linguistic measures (Table 5): word count, H(4) = 95.08, p < 0.001, ε² = 0.959; MATTR-50, H(4) = 68.33, p < 0.001, ε² = 0.677; mean sentence length, H(4) = 90.70, p < 0.001, ε² = 0.913; MLT, H(4) = 88.83, p < 0.001, ε² = 0.893; DC/T, H(4) = 88.46, p < 0.001, ε² = 0.889; and CN/C, H(4) = 58.00, p < 0.001, ε² = 0.568.

Table 5. Linguistic differentiation across intended CEFR levels in the MA condition.

Measure	Kruskal–Wallis H(4)	p	ε2	Spearman ρ	p	Significant Adjacent-Level Differences
Word count	95.08	<0.001	0.959	0.980	<0.001	A1–A2; A2–B1; B1–B2; B2–C1
MATTR-50	68.33	<0.001	0.677	0.805	<0.001	A2–B1; B1–B2
MSL	90.70	<0.001	0.913	0.950	<0.001	A1–A2; A2–B1; B1–B2; B2–C1
MLT	88.83	<0.001	0.893	0.947	<0.001	A1–A2; A2–B1; B1–B2; B2–C1
DC/T	88.46	<0.001	0.889	0.920	<0.001	A1–A2; A2–B1; B1–B2
CN/C	58.00	<0.001	0.568	0.741	<0.001	B1–B2; B2–C1

Note: Analyses include MA-generated outputs only (N = 100; n = 20 per intended CEFR level). CEFR level was treated as an ordered variable from A1 to C1 for Spearman rank correlations. ε² = epsilon-squared effect size; MSL = mean sentence length; MLT = mean length of T-unit; DC/T = dependent clauses per T-unit; CN/C = complex nominals per clause. Adjacent-level comparisons were conducted using Mann–Whitney U tests with Holm adjustment across the four adjacent-level comparisons within each linguistic measure. Only statistically significant comparisons (adjusted p < 0.05) are shown.

Spearman rank correlations showed positive monotonic associations between the ordered intended CEFR level and each linguistic measure. The association was highest for word count (ρ = 0.980), followed by mean sentence length (ρ = 0.950), MLT (ρ = 0.947), DC/T (ρ = 0.920), MATTR-50 (ρ = 0.805), and CN/C (ρ = 0.741); all correlations were statistically significant (p < 0.001). These associations indicate that higher intended proficiency levels were generally accompanied by longer texts, greater lexical diversity, and higher values on the examined syntactic measures within the MA condition. However, the particularly strong association for word count should be interpreted in light of the predefined level-specific story-length ranges, which directly influenced this outcome. More generally, these correlations describe linguistic patterns across intended levels rather than independently validating CEFR appropriateness.

Holm-adjusted pairwise comparisons between adjacent intended CEFR levels provided a more detailed view of this differentiation. Word count, MSL, and MLT differed significantly at every successive transition from A1 to C1. Lexical diversity, as measured by MATTR-50, showed significant differences from A2 to B1 and from B1 to B2, whereas the A1–A2 and B2–C1 differences were not statistically significant. In particular, the absence of a significant MATTR-50 difference between B2 and C1 indicates that lexical diversity did not provide clear adjacent-level discrimination at the upper end of the examined proficiency range. DC/T differed significantly from A1 through B2 but not between B2 and C1 (p = 0.957). In contrast, CN/C did not differ significantly between A1 and A2 or between A2 and B1, whereas significant differences emerged from B1 to B2 (p < 0.001) and from B2 to C1 (p = 0.003).

Taken together, these findings show systematic linguistic differentiation across the intended proficiency levels within the MA condition, with particularly consistent level-wise differences in text length and sentence/T-unit length. However, adjacent-level differentiation was not uniform across all measures, especially for lexical diversity and specific dimensions of syntactic complexity. Furthermore, these findings characterize the linguistic properties of MA-generated outputs and do not establish that MA provides stronger proficiency differentiation than CP or that the generated texts independently satisfy CEFR proficiency standards.

3.3. Comparison of PB, CP, and MA Generation

Matched comparisons were conducted among PB, CP, and MA outputs generated for the same task and intended CEFR level. Friedman tests were first used to examine overall differences among the three generation conditions for each linguistic measure within each proficiency level. Significant overall differences were observed for most text-length and syntactic measures, whereas differences in lexical diversity were comparatively limited. The magnitude of the overall condition differences, quantified using Kendall’s W, varied substantially across measures and proficiency levels (Table 6).

Table 6. Friedman tests comparing PB, CP, and MA outputs by intended CEFR level.

CEFR	Measure	χ2(2)	p	Kendall’s W
A1	Word count	0.545	0.761	0.014
	MATTR-50	3.700	0.157	0.093
	MSL	33.443	<0.001	0.836
	MLT	32.430	<0.001	0.811
	DC/T	24.033	<0.001	0.601
	CN/C	2.364	0.307	0.059
A2	Word count	40.000	<0.001	1.000
	MATTR-50	7.900	0.019	0.198
	MSL	10.900	0.004	0.273
	MLT	19.900	<0.001	0.498
	DC/T	1.000	0.607	0.025
	CN/C	4.900	0.086	0.123
B1	Word count	31.600	<0.001	0.790
	MATTR-50	1.300	0.522	0.033
	MSL	30.100	<0.001	0.753
	MLT	27.300	<0.001	0.683
	DC/T	26.051	<0.001	0.651
	CN/C	15.700	<0.001	0.393
B2	Word count	24.700	<0.001	0.617
	MATTR-50	0.300	0.861	0.008
	MSL	33.600	<0.001	0.840
	MLT	33.600	<0.001	0.840
	DC/T	10.101	0.006	0.253
	CN/C	16.442	<0.001	0.411
C1	Word count	36.400	<0.001	0.910
	MATTR-50	5.200	0.074	0.130
	MSL	3.700	0.157	0.093
	MLT	9.300	0.010	0.233
	DC/T	10.900	0.004	0.273
	CN/C	1.600	0.449	0.040

Note: Friedman tests were based on 20 matched tasks per intended CEFR level, with each task represented under PB, CP, and MA. Kendall’s W is reported as the effect-size measure. PB = prompt-based generation; CP = constrained prompting; MA = multi-agent generation; MSL = mean sentence length; MLT = mean length of T-unit; DC/T = dependent clauses per T-unit; CN/C = complex nominals per clause.

For word count, significant overall differences among the three conditions were observed at A2, B1, B2, and C1, but not at A1. The largest condition differences occurred at A2 (W = 1.000) and C1 (W = 0.910). At A2 and B1, PB produced the longest texts, followed by MA and CP. At B2, PB also produced the longest texts, whereas CP was slightly longer than MA. At C1, this pattern changed substantially: CP produced the longest texts (M = 520.75), followed by MA (M = 489.10), while PB produced considerably shorter texts (M = 315.25). These differences should be interpreted in light of the explicit level-specific story-length requirements provided to CP and MA but not to PB. The unusually low C1 word count in PB represents a departure from the pattern observed at lower levels; the present experiment does not establish why this occurred.

Lexical diversity showed substantially smaller differences among the three conditions. Friedman tests for MATTR-50 were statistically significant only at A2 (p = 0.019), while the omnibus tests at A1, B1, B2, and C1 were not significant. This pattern suggests that the three generation conditions generally produced similar levels of lexical diversity despite clearer differences in text length and syntactic characteristics.

Condition differences were more pronounced for sentence-level and syntactic measures. MSL differed significantly among the three conditions at A1, A2, B1, and B2, but not at C1. MLT showed significant overall differences at all five intended proficiency levels. DC/T differed significantly at A1, B1, B2, and C1, whereas the A2 comparison was not significant. CN/C differed significantly at B1 and B2, while the omnibus comparisons at A1, A2, and C1 were not statistically significant.

The three prespecified paired contrasts provided a more detailed assessment of these differences. Across five intended CEFR levels and six linguistic measures, 30 level–measure combinations were evaluated for each of the three planned contrasts (PB-CP, PB-MA, and CP-MA), resulting in 90 planned pairwise comparisons in total. Significant Holm-adjusted differences were identified in 18 of 30 comparisons for PB-CP, 20 of 30 for PB–MA, and 12 of 30 for CP-MA. Thus, PB differed from both constrained conditions more frequently than CP and MA differed from one another.

The 12 significant CP-MA planned contrasts are presented in Table 7. These differences were distributed across all five intended CEFR levels and involved word count and selected syntactic measures, whereas no significant CP–MA differences were observed for MATTR-50. MA produced significantly higher values in five comparisons, including MSL and DC/T at A1, word count and MSL at A2, and word count at B1. Conversely, CP produced significantly higher values in seven comparisons, including CN/C at A2, word count, MSL, and MLT at B2, and word count, MLT, and DC/T at C1. The rank-biserial effect sizes indicate that several of these differences were substantial in magnitude. However, higher values on these linguistic measures do not necessarily indicate greater pedagogical suitability or CEFR appropriateness. Moreover, because CP and MA shared core generation requirements and vocabulary resources but differed in instructional detail, the observed differences cannot be attributed exclusively to architectural organization.

Table 7. Significant CP-MA paired comparisons by intended CEFR level.

CEFR	Measure	CP Mean	MA Mean	Difference (MA − CP)	Holm-Adjusted p	Rank-Biserial r
A1	MSL	8.01	8.74	+0.73	0.008	−0.689
A1	DC/T	0.004	0.033	+0.029	0.018	−1.000
A2	Word count	148.35	161.45	+13.10	<0.001	−1.000
A2	MSL	11.98	13.24	+1.26	<0.001	−0.857
A2	CN/C	0.756	0.626	−0.130	0.046	+0.610
B1	Word count	244.35	255.95	+11.60	0.004	−0.714
B2	Word count	339.35	326.30	−13.05	0.033	+0.548
B2	MSL	21.43	20.48	−0.95	0.024	+0.571
B2	MLT	17.82	16.88	−0.94	0.009	+0.648
C1	Word count	520.75	489.10	−31.65	<0.001	+0.800
C1	MLT	24.16	21.43	−2.73	0.008	+0.705
C1	DC/T	1.143	0.886	−0.257	0.004	+0.781

Note: Only statistically significant CP–MA planned contrasts (Holm-adjusted p < 0.05) are reported. Mean differences were calculated as MA − CP. Positive differences indicate higher mean values under MA, whereas negative differences indicate higher mean values under CP. Rank-biserial correlations were calculated using paired differences defined as CP − MA; therefore, negative correlations indicate higher ranked values under MA, while positive correlations indicate higher ranked values under CP. Holm adjustment was applied across the three prespecified contrasts within each CEFR-level × linguistic-measure family. Each comparison included 20 matched tasks.

Overall, the results indicate that explicit level-specific constraints and resources are associated with substantial differences between conventional prompting and the two controlled generation approaches. The smaller number of significant CP–MA contrasts suggests greater similarity between CP and MA, although the remaining differences were level- and measure-dependent. Because the CP prompts and MA agent configurations were not identical in instructional detail, these differences cannot be attributed exclusively to architectural organization. Accordingly, the findings do not indicate a uniform linguistic advantage of MA over constrained prompting; rather, they identify selected characteristics for which the two implementation approaches produced different outputs.

3.4. Sensitivity Analysis of Lexical Diversity

As a sensitivity analysis, lexical diversity was recalculated using MATTR-25. MATTR-25 was strongly correlated with the primary MATTR-50 measure across all 300 outputs (Spearman’s ρ = 0.955, p < 0.001). Three-condition comparisons using MATTR-25 showed limited overall differences, with a significant Friedman test only at C1 (χ²(2) = 6.700, p = 0.035, W = 0.168). Among the Holm-adjusted planned contrasts, only the C1 PB–MA comparison was significant (p = 0.022, rank-biserial correlation = −0.667), while no CP–MA contrast reached significance at any intended proficiency level.

Although the exact significance pattern differed from MATTR-50, the strong association between the two MATTR measures and the limited number of significant condition contrasts suggest that the overall interpretation was not substantially altered by the choice of moving-window size. Nevertheless, the sensitivity analysis indicates that individual statistical comparisons may be sensitive to the lexical-diversity window selected, and the findings should therefore not be interpreted as demonstrating complete agreement between MATTR-50 and MATTR-25.

3.5. Operational Constraint Adherence

Operational constraint adherence was examined across the 100 CP and 100 MA outputs using three objectively verifiable requirements shared by the two conditions: story-length range, target-vocabulary count, and comprehension-question count. Complete adherence was observed for all three requirements at every intended CEFR level in both conditions: all 100 CP outputs and all 100 MA outputs satisfied the corresponding story-length range, required target-vocabulary count, and prescribed number of comprehension questions.

These adherence rates refer to the final outputs included in the analysis. The workflow did not include an automatic retry or regeneration mechanism, and all 100 final outputs in each condition were included in the adherence analysis. However, because complete execution logs were not retained, first-attempt success rates could not be independently verified. Thus, complete adherence was not specific to the multi-agent architecture; rather, both implementation approaches satisfied the explicit operational constraints in the analyzed outputs. These results represent compliance with author-defined generation requirements and should not be interpreted as independent evidence of CEFR appropriateness.

4. Discussion

The findings of this study contribute to the emerging literature on agentic AI for educational content generation by examining proficiency-oriented linguistic differentiation under three generation conditions: conventional prompting (PB), constrained prompting (CP), and orchestrated multi-agent generation (MA). While recent studies have explored specialized LLM agents for course-material generation, personalized educational content, intelligent tutoring, and automated assessment development [7,8,15,16,17,18,19], the present study focuses on a different problem: controlling the linguistic characteristics of generated materials across five intended CEFR proficiency levels using the same underlying language model and matched generation tasks.

Within the MA condition, all six linguistic measures differed significantly across intended proficiency levels, although adjacent-level differentiation varied across measures. The three-condition comparison further demonstrated that PB differed more frequently from both controlled generation approaches than CP and MA differed from one another. Across the 30 planned level-by-measure comparisons, significant differences were observed in 18 PB-CP comparisons, 20 PB-MA comparisons, and 12 CP-MA comparisons. Both CP and MA also satisfied all three explicitly specified operational requirements in the analyzed final outputs.

The detailed CP–MA comparisons provide an important qualification to the architectural contribution of the proposed approach. Of the 12 significant contrasts, MA produced higher values in five, whereas CP produced higher values in seven. These differences involved text length and selected syntactic measures, with no significant CP–MA differences in MATTR-50. Importantly, higher values do not necessarily indicate better educational content, particularly when the objective is to produce materials appropriate for a specified proficiency level. Furthermore, although CP and MA shared core generation requirements and vocabulary resources, their complete instructional configurations were not identical. Consequently, the observed differences cannot be attributed exclusively to architectural organization.

Taken together, the findings suggest that explicit proficiency-specific requirements and vocabulary resources play an important role in controlled generation. The contribution of the MA approach is therefore better understood as providing a modular mechanism for organizing and applying these configurations, accompanied by selected linguistic differences, rather than demonstrating general linguistic superiority over constrained prompting.

4.1. CEFR-Level Differentiation and Generation Control

The findings reinforce the view that proficiency-controlled language-learning content generation is a multidimensional problem rather than a task that can be addressed solely by specifying a CEFR label in a prompt. Recent research has similarly highlighted the challenges of maintaining fine-grained control over CEFR-oriented materials. Zuo [3], for example, investigated CEFR-specific ESL material generation through explicit linguistic features, optimization, and controlled generation mechanisms. Similarly, the TSAR 2025 Shared Task on readability-controlled text simplification highlighted the use of structured generation strategies, including iterative refinement and multi-agent configurations, for targeting different readability levels [20,21].

The results for RQ1 are consistent with this multidimensional perspective. Although all six linguistic measures showed significant positive associations with increasing intended proficiency level in the MA condition, adjacent-level comparisons revealed different patterns of differentiation. Word count, mean sentence length (MSL), and mean length of T-unit (MLT) distinguished consecutive levels more consistently, whereas lexical diversity and specific syntactic-complexity measures exhibited more selective patterns. In particular, dependent clauses per T-unit (DC/T) differentiated several lower and intermediate level transitions but did not significantly distinguish B2 from C1. Conversely, complex nominals per clause (CN/C) showed significant differences primarily at the higher proficiency levels.

These findings suggest that proficiency-oriented differentiation should not be conceptualized as a uniform increase in every linguistic characteristic. Different dimensions may contribute to distinctions between intended proficiency levels in different ways. Nevertheless, the observed patterns must be interpreted in relation to the generation configurations. Story-length ranges, vocabulary requirements, and grammatical expectations were explicitly defined for each level. Consequently, some aspects of the observed progression, particularly word count, were partly determined by the experimental design rather than emerging independently from the generation architecture.

The comparison among PB, CP, and MA further illustrates why greater linguistic complexity should not automatically be interpreted as better proficiency alignment. PB frequently produced higher values on selected syntactic measures, including MSL, MLT, DC/T, and CN/C, without consistently following the ordered progression observed under the controlled generation conditions. The relatively short PB outputs at C1 provide another example of this variability. However, the present experiment does not establish the cause of this pattern, and it should not be interpreted as evidence that PB inherently fails at advanced-level generation. Several factors may explain the decrease in PB story length at C1. First, the PB prompt did not impose explicit length constraints, potentially allowing the model to prioritize linguistic complexity over text length. Second, the model may have interpreted the advanced proficiency level differently in terms of expected content and structure. Third, stochastic variability associated with generating only one output per task may have contributed to this pattern. These explanations remain hypothetical, as the present study did not investigate the underlying causes.

More broadly, the findings highlight the distinction between measurable linguistic characteristics and pedagogical appropriateness. Text length, lexical diversity, and syntactic complexity provide useful indicators for examining generated materials, but they do not independently establish whether a text is suitable for learners at a particular CEFR level. This distinction is especially relevant given recent work suggesting that increasingly capable readability-controlled generation systems can challenge the discriminative capacity of existing automatic evaluation metrics [20,21,22]. Accordingly, the present results support conclusions about systematic linguistic differentiation under explicitly configured generation conditions, while independent educational validation remains necessary to establish CEFR appropriateness.

4.2. Conceptual and Methodological Contributions

This study makes three main contributions to research on generative AI for language-learning content.

First, it extends the application of agent specialization to proficiency-oriented educational content generation. Existing educational multi-agent systems frequently distribute responsibilities across functional roles or workflow stages, such as content development, tutoring, assessment, and feedback [8,23,24,25]. In contrast, the proposed architecture organizes specialization around intended CEFR proficiency levels within the same generation task. A centralized orchestrator routes each request to a corresponding level-specific agent, which applies its configured instructions and vocabulary resources.

This arrangement represents orchestrated specialization rather than collaborative multi-agent reasoning. The agents do not negotiate, critique one another, or jointly construct an output. Instead, the architecture separates proficiency-specific configurations into independently organized components. Its conceptual contribution therefore lies in demonstrating how proficiency specialization can be represented through modular agent configurations without requiring different underlying language models.

Second, the study operationalizes proficiency-targeted generation through a combination of explicit instructional requirements and measurable output characteristics. Rather than relying exclusively on CEFR labels, the generation configurations specify story-length ranges, vocabulary targets, grammatical expectations, and comprehension activities. The evaluation then examines both adherence to selected operational requirements and variation in linguistic characteristics across intended proficiency levels.

This approach distinguishes configured generation requirements from observed linguistic outcomes. The distinction is methodologically important because satisfying an author-defined requirement does not necessarily establish that the resulting content meets an externally validated educational standard. By examining multiple linguistic measures alongside operational adherence, the study provides a more structured basis for analyzing proficiency-oriented generation than reliance on a single readability indicator.

Third, the study introduces a matched three-condition comparison that distinguishes conventional prompting from two implementations of explicitly constrained generation. PB, CP, and MA used the same underlying language model and the same 100 level–topic tasks. The inclusion of CP provides a particularly important comparison because it applies core level-specific requirements and vocabulary resources without using the orchestrator or specialized agent routing.

This design enables the contribution of explicit prompting constraints to be examined separately from the additional differences observed under MA. However, because the CP prompts and MA agent instructions were not identical in all details, the comparison does not isolate architectural organization as a single causal factor. The methodological contribution is therefore a more informative comparison of three generation approaches, rather than a controlled demonstration of the independent causal effect of multi-agent architecture.

Together, these contributions position the study as both an empirical comparison of proficiency-oriented generation approaches and a descriptive investigation of a modular architecture for organizing level-specific generation requirements.

4.3. Practical Implications

The findings have practical implications for the design of AI-supported language-learning platforms, particularly those requiring content generation across multiple proficiency levels.

One implication concerns the representation and management of generation requirements. In the proposed architecture, proficiency-specific instructions and vocabulary resources are organized into separate agent configurations. A platform can route a generation request to the relevant component based on the intended proficiency level, allowing the same underlying language model to operate under different instructional configurations.

This design may be useful in systems where proficiency requirements change over time or where additional levels, content types, or vocabulary resources must be incorporated. For example, developers could revise the instructions associated with a particular proficiency level without necessarily modifying the configurations used for other levels. Such modularity offers a potential engineering advantage in terms of configuration management and extensibility. However, these benefits remain architectural expectations rather than empirically measured improvements in maintenance effort, scalability, or operational efficiency.

A second implication concerns the role of constrained prompting. The results show that direct constrained prompting can satisfy the same operational requirements as the orchestrated MA implementation. Therefore, systems that primarily require compliance with predefined story-length, vocabulary-count, and question-count specifications may not necessarily need a multi-agent architecture. The choice between CP and MA should instead consider the complexity of the application, the number of configurations to be managed, and the potential value of separating proficiency-specific instructions into modular components.

A third implication concerns the integration of automated generation with educational quality assurance. Although both controlled approaches satisfied the specified operational requirements in the analyzed outputs, such compliance does not establish that the generated stories are pedagogically appropriate. In practical applications, automated checks for measurable requirements could be combined with expert review of vocabulary suitability, grammatical appropriateness, coherence, comprehension-question quality, and alignment with instructional objectives.

Finally, the architectural pattern could potentially be extended beyond story generation. The same routing principle could support dialogues, vocabulary exercises, reading activities, and assessment materials by introducing task- and proficiency-specific configurations. Such extensions would require additional implementation and evaluation, particularly when the instructional objectives differ substantially from those examined in the present study.

4.4. Limitations and Future Research

Several limitations should be considered when interpreting the findings. First, the study used a single underlying language model and a predefined set of English story-generation tasks, limiting generalizability across models, languages, and instructional formats. Furthermore, although CP and MA shared core generation requirements and vocabulary resources, their instructional configurations were not identical. Consequently, the observed differences cannot be attributed exclusively to architectural organization. Future studies could compare multiple language models and employ factorial designs that independently manipulate instructional specificity and architectural implementation.

Second, the experimental design did not assess variability across repeated generations. Moreover, the available execution records were insufficient to independently verify all intermediate attempts or establish first-attempt success rates. Therefore, the complete operational adherence observed in the analyzed CP and MA outputs should not be interpreted as evidence of consistent first-pass reliability. Accordingly, the 12 statistically significant CP–MA contrasts should be interpreted as findings specific to the analyzed generation runs rather than as stable estimates of differences across repeated generations. Future research should incorporate repeated generations, complete execution logs, and systematic reporting of retries and exclusions.

Third, the proficiency-specific configurations relied on author-defined linguistic requirements and vocabulary resources rather than independently validated CEFR mappings. The relatively small C1 vocabulary inventory (127 entries) may also have limited lexical selection at the advanced level. Additionally, the evaluation relied primarily on automated linguistic measures and operational constraint adherence, without independent expert assessment of CEFR appropriateness. Future studies should employ documented and validated vocabulary resources, blinded expert evaluations, and learner-based assessments to establish the pedagogical relevance of the generated materials.

Finally, the evaluation of comprehension activities focused primarily on question counts rather than their cognitive demand, answerability, or instructional quality. Future research could extend the evaluation to these dimensions and investigate whether the proposed architecture supports other educational content types, including dialogues, vocabulary exercises, and assessment materials. Such studies could also examine the practical implications of modular agent configurations in terms of maintainability, scalability, and development effort.

5. Conclusions

This study investigated an orchestrated multi-agent AI architecture for generating CEFR-oriented language-learning content and compared it with conventional prompt-based (PB) and constrained prompting (CP) generation using the same underlying language model and 100 matched tasks across five intended CEFR proficiency levels.

The results demonstrated systematic linguistic differentiation across A1–C1 within the MA condition, although differentiation was not uniform across linguistic dimensions or adjacent proficiency levels. In the three-condition comparison, PB differed more frequently from both controlled generation approaches, while CP and MA showed greater similarity. Nevertheless, 12 of the 30 planned CP–MA comparisons revealed statistically significant differences, with neither approach consistently producing higher values across linguistic measures. Both CP and MA also satisfied all three explicitly specified operational requirements in the analyzed final outputs. These findings highlight the importance of explicit proficiency-specific constraints and vocabulary resources while indicating that the observed differences between CP and MA do not establish a general linguistic advantage for multi-agent generation.

The proposed architecture therefore contributes a modular approach to organizing and applying proficiency-specific generation configurations through centralized orchestration and specialized agents. However, because the CP and MA instructional configurations were not identical, their observed differences cannot be attributed exclusively to architectural organization. Moreover, systematic linguistic differentiation and operational constraint adherence do not independently establish CEFR appropriateness or pedagogical effectiveness. Future research should investigate the approach across different models and educational contexts, with repeated generations, independent expert evaluation, and learner-based validation.