Beyond Answers: Pedagogical Design Rationale for Multi-Persona AI Tutors

Abstract

This paper reports a design-rationale account of building and deploying a small ecosystem of AI-driven educational conversational agents with distinct pedagogical personas. Two strands target school contexts: (i) Talk to Bill, a historically grounded Shakespeare interlocutor intended to support close reading, contextual understanding, and interpretive dialogue; and (ii) Here to Help, a set of UK GCSE subject- and exam-board-specific tutors designed for formative practice in recognised question formats with feedback and iterative improvement. The third strand comprises six complementary assistants for an undergraduate Human–Computer Interaction (HCI) module, each bounded to a workflow-aligned role (e.g., empathise-stage coaching, study planning, course operations), with guardrails to privilege process quality over answer generation. We describe how persona differentiation is mapped to established learning, engagement, and motivation theories; how retrieval-augmented generation and provenance cues are used to reduce hallucination risk; and what early deployment observations suggest about orchestration, integration, and incentives. The contribution is a transferable, auditable rationale linking theory to concrete dialogue and UI moves for multi-persona tutoring ecosystems, rather than a claim of causal learning gains.

1. Introduction

Large language models (LLMs) are increasingly embedded in learning workflows, yet their effectiveness and acceptability depend on how they are framed, scaffolded, and integrated into assessment and teaching practice. Meta-analytic evidence suggests intelligent tutoring and structured support can yield medium-to-large learning gains [1,2], and experimental syntheses on generative AI report small-to-moderate positive effects, tempered by wide heterogeneity across tasks, prompts, and guardrails [3].

This paper contributes a design account and early use observations for three systems:

• Talk to Bill (schools): a Shakespeare-inspired agent designed to promote close reading, historical context, and interpretive dialogue for pupils.
• Here to Help (schools): a set of exam board and subject specific agents focussed on supporting UK GCSE students
• Human-Computer Interaction course assistants (University): six persona-based agents aligned to complementary pedagogical roles for an HCI module.

The three strands share a common ”beyond answers” aim—supporting learning processes rather than delivering finished products—but differ in (a) learner population (KS3–4; GCSE; undergraduate), (b) primary pedagogical mechanism (dialogic interpretation; criterion-referenced practice; workflow-aligned coaching), and (c) integrity/provenance strategy (persona boundaries; exam-format constraints + RAG; role-bounded assistants + RAG and policy guardrails). We therefore present (i) a common theoretical and design rationale (Section 2, Section 3 and Section 4), then (ii) strand-specific instantiations (Section 5, Section 6 and Section 7), and finally (iii) cross-cutting adoption observations on orchestration and integration (Section 9 and Section 10).

Scope, Evidence Type, and Positionality

This article is intentionally framed as a design-rationale and deployment report. Its goal is to make the design logic auditable: how pedagogical theory, human-AI interaction principles, and institutional constraints are translated into concrete dialogue policies, prompts, guardrails, and interface affordances. The paper does not claim causal learning gains or present controlled intervention experiments; where outcomes are discussed, they are limited to early deployment observations, informal feedback, and adoption barriers that motivate future evaluation.

Author role and positionality. The systems described here were designed and developed by the author in collaboration with local technical support. This proximity provides access to fine-grained implementation detail, but also introduces potential bias (e.g., selective attention to positive cases). To mitigate this, the paper (i) states the boundaries of evidence explicitly, (ii) reports adoption frictions alongside positive feedback, and (iii) provides reproducibility-oriented artefacts (corpus inventories, configuration parameters, logging schema) sufficient for independent scrutiny (Section 4).

Our focus is on describing the rationale for giving agents different pedagogical approaches and personalities, how these map to learning theories, and why such differentiation matters for student experience, trust, and outcomes. We also report on the uptake and user responses to these systems: some are viewed very positively, whilst others, despite being helpful for revision and formative practice, have seen limited use at this early point in the module and we explore design and orchestration shifts to address this.

2. Background and Related Work

2.1. One-to-One Tutoring and Its Implications for AI Agents

Classic work on tutoring foregrounds the potential of one-to-one support to substantially improve attainment. Bloom’s “2 sigma problem” [4] aggregated controlled comparisons among three conditions: conventional whole-class instruction, mastery learning with systematic formative assessment and corrective enrichment, and individual tutoring (one tutor working with one learner) layered on top of mastery learning. On average, students receiving one-to-one tutoring performed around two standard deviations above those in conventional classes, placing the average tutored student near the 98th percentile of the control group; mastery-learning cohorts often achieved gains approaching one standard deviation over conventional instruction. Bloom argued that the central features enabling these gains were tight feedback loops, corrective instruction targeted to the learner’s specific misconceptions, opportunities to practice to mastery, and the social-motivational affordances of persistent personalised attention [4,5]. Subsequent syntheses nuance this picture. Meta-analyses and comparative reviews conclude that well-designed Intelligent Tutoring Systems (ITS) can approach the effectiveness of human tutoring in some domains and settings, though human tutors still tend to hold an advantage [1,2,6]. Natural-language dialogue tutors such as AutoTutor demonstrate measurable learning gains by combining deep-reasoning prompts, contingent feedback, and graduated scaffolding [7].

The implications for AI tutoring agents are twofold. First, the primary design target is not merely content delivery but the recreation (or approximation) of the mechanisms Bloom identified: rapid, diagnostic feedback keyed to the individual’s evolving state; corrective and enrichment activities aligned to specific errors; opportunities to practise to criterion; and sustained, socially credible encouragement. Second, because learners differ in their needs and goals, agent behaviours should be intentionally differentiated rather than homogenised: a single “one-size-fits-all” conversational style risks averaging away the benefits of adaptivity that drive the 2 sigma effect.

Operationalising Bloom’s 2σ

We operationalise the insights in Bloom’s approach through the mechanisms in

Table 1. Bloom mechanisms mapped to design moves and agent coverage. “Arby” is the HCI teaching–assistant agent (Section 7.4 and “Shelby” is the study support and wellbeing coach (Section 7.5).

Mechanism	Design Move (Instantiation)	Agents	Process Trace (Intended)
Tight feedback	Step-based hints; success criteria; refusal to produce deliverables	Arby	Fewer hint tiers over sessions; ↑ first-attempt correctness
Corrective instruction	Method trade-offs; evidence ledger; ethics prompts	Emmy	↑ justification tokens; completed checklists
Mastery practice	Retrieval bursts; spaced plans; revision sprints	Arby, Shelby	Scheduled revisits; stable time-on-task
Sustained encouragement	Autonomy-supportive choices; empathic acknowledgements	Shelby	↑ voluntary re-engagement; reduced deflection

. This is intended as an “audit bridge” between theory and implementation. It lists Bloom-aligned tutoring mechanisms (e.g., rapid diagnosis, corrective instruction, mastery practice) and the concrete dialogue/UI moves used to instantiate them in each agent. “Coverage” indicates which agents implement the mechanism as a first-class routine (not merely incidental capability).

2.2. Evidence from Recent GenAI Tutors and Assistants

A fast-growing empirical literature evaluates generative AI (GenAI) systems as tutors, feedback providers, or programming partners. Two recent meta-analyses synthesising experimental and quasi-experimental studies report overall positive effects of ChatGPT-like tools on academic performance (large effects), with moderate positive effects on affective-motivational outcomes and higher-order thinking, albeit with heterogeneity across settings, durations, and instructional roles [3,8]. These aggregated results are consistent with earlier ITS evidence [1,6] and with dialogue-based tutoring approaches [7], while also highlighting boundary conditions: effects tend to be stronger when GenAI is embedded in explicit pedagogical frames (e.g., problem-based learning, mastery sequences) and used over multi-week periods [3,8].

In programming education specifically, findings are mixed but increasingly granular. Controlled classroom experiments and case studies show that GenAI can accelerate task completion and improve product quality under certain conditions, but can also induce over-reliance or superficial understanding if unscaffolded. For example, in an in-class supported exercise education track study, adding ChatGPT assistance for introductory programming tasks did not reliably improve student outcomes and sometimes degraded style or reasoning without structured guidance [9]. By contrast, quasi-experimental studies of AI-assisted pair programming report higher motivation, reduced programming anxiety, and better performance relative to individual work and in some cases relative to human-human pairing [10]. Evaluations of GenAI-supported feedback in University programming courses indicate that LLM chatbots can generate timely, actionable comments aligned with instructor rubrics, which students use to iterate toward higher-quality solutions [11]. Beyond programming, studies and meta-reviews of LLM-generated formative feedback in writing and other domains also show performance gains when feedback is specific, explanatory, and tied to criteria, with effects moderated by duration and integration into assessed practice [3,8].

2.3. Cognitive Engagement and Instructional Design

By using agent-based systems, conversational interaction can be engineered to elicit deeper forms of cognitive engagement than monologic presentation. The ICAP framework predicts a monotonic relationship between the quality of overt engagement and learning outcomes: interactive activities (co-construction and negotiation) outperform constructive (self-generation beyond given information), which outperform active (manipulation or selection), which outperform passive (listening or reading) [12,13]. Dialogue with an agent affords rapid transitions among these engagement modes. For example, strategically prompting learners to generate explanations, compare alternative solutions, or repair misconceptions moves activity from active to constructive; turn-taking that requires learners to justify and critique moves it further into the interactive regime [13].

Cognitive Load Theory (CLT) complements this by constraining how explanations and tasks are sequenced and represented so that intrinsic load is matched to prior knowledge, extraneous load is minimised, and germane load is reserved for schema construction [14]. In agent design, this implies the use of worked examples and example-problem pairs early on, with fading of scaffolds as competence increases; split-attention and redundancy effects are mitigated by multi-turn decomposition that keeps the focus of attention narrow while retaining a global plan [14]. When guidance is removed too soon or too completely, minimally guided discovery tends to underperform for novices [15]. Accordingly, our agents implement graduated guidance with explicit criteria for when to fade, and with contingent re-scaffolding when errors or disfluency indicate overload.

Conversational tutoring literature provides fine-grained mechanisms for turning general principles into dialogue moves. Naturalistic analyses of one-to-one tutoring show that effective sessions feature collaborative problem solving, deep explanatory reasoning, and tightly coupled feedback cycles that converge toward shared meanings [16]. Controlled studies comparing tutorial dialogue (both human and computer-mediated) with reading show sizeable benefits when content is kept constant and the dialogue is well aligned with learner preparation; effects are largest when there is a mismatch between text difficulty and prior knowledge that the dialogue can repair [17]. These findings motivate agent behaviours such as opportunistic self-explanation prompts, hypothesis-testing questions, and negotiated definitions that make knowledge gaps explicit and actionable.

Two additional strands further strengthen the case for conversation. First, the feedback literature suggests that targeted, timely, and actionable feedback has among the highest instructional payoffs, particularly when it reduces uncertainty about goals, progress, and next steps [18]. Conversational agents can deliver this rapidly, in context, and with adaptive specificity. Second, retrieval practice improves long-term retention over restudy, especially when tests are spaced and feedback is provided [19]. Agents can interleave micro-quizzes with elaborative follow-up, turning dialogue into distributed retrieval practice rather than continuous exposition. Finally, when explanations require generative activity such as summarising, analogising, or teaching back, learning benefits accrue beyond exposure to worked solutions [20].

We therefore implement dialogue policies that deliberately elicit constructive and interactive moves, calibrate task complexity to manage cognitive load, and blend explanation with questioning and practice. Scaffolding is kept contingent, faded over time, and accompanied by transfer of responsibility to the learner [21]. These choices give a principled account of how conversational interaction improves cognitive engagement and, by design, should yield stronger learning than passive or minimally guided alternatives.

2.4. Motivation, Agency, and Identity

Motivational design in agents can be grounded in complementary theories. Self-Determination Theory (SDT) proposes that autonomy, competence, and relatedness underpin high-quality motivation; environments that support these needs enhance persistence and performance [22]. In practice, persona and tone influence perceived relatedness and psychological safety; choice over task order or strategy supports autonomy; and graduated challenge with clear criteria for success supports competence. Expectancy-Value theory further predicts that engagement depends on learners’ beliefs about success and the value of the task (attainment, utility, interest) weighed against perceived cost [23]. Dialogue affords continuous calibration of expectancies (by surfacing proximal goals and visible progress) and task value (by contextualising problems and connecting to learner interests). Self-efficacy—beliefs about one’s capability to succeed - functions as a proximal predictor of effort and resilience, and targeted mastery experiences and attributional feedback within dialogue can raise self-efficacy [24]. Finally, identity-relevant framing and process-focused messages can shift beliefs about ability and trajectories of effort; population-level studies indicate that brief, well-targeted interventions can improve achievement under specific conditions [25].

Our agent personas are therefore instrumented to target different motivational levers. A highly structured, mastery-oriented persona emphasises competence through clear learning progressions and criterion-referenced feedback. A coaching persona emphasises autonomy through planning choices and reflective prompts. A warm, mentoring persona emphasises relatedness through affect-aware responses and inclusive language. By diversifying personas, we increase the chance that at least one perceived relationship “fits” the student’s preferences and context, while preserving a shared set of evidence-based instructional moves [26].

2.5. Pedagogical Theories Operationalised in the Agents

We utilise a number of pedagogical theories within our agent design process, and these are summarised here.

Socratic elicitation and tutorial dialogue provides a well-known approach though questioning the interlocutor rather than providing answers. Socratic tutoring aims to surface contradictions, elicit warrants, and prompt self-repair rather than deliver answers. Empirical work on tutorial dialogue and intelligent tutors shows that deep-reasoning questions and mixed-initiative exchanges are associated with better learning, particularly when content coverage is controlled [16,17]. The agents therefore maintain a bias for asking why/how questions, requesting predictions, and negotiating criteria before supplying canonical solutions.

Laurillard’s Conversational Framework explains that learning progresses through iterative cycles between concepts and practice, with feedback on both, mediated by teacher-learner and peer conversations [27]. The agents instantiate these cycles by alternating conceptual modelling (worked and partially worked explanations) with practice generation (problems, projects), each followed by feedback that links actions to conceptual models. Dialogue embeds short, low-stakes retrieval opportunities with immediate, informative feedback to consolidate learning and calibrate metacognition [18,19]. Spacing and interleaving can be scheduled across sessions to extend retention gains beyond the immediate conversation.

Instruction is most effective when it targets the learner’s Zone of Proximal Development (ZPD) [28], the space between independent performance and performance achievable with guidance. Operationally, this implies contingent support that is gradually withdrawn as competence grows and responsibility transfers to the learner [21,29]. The agents estimate ZPD boundaries using recent dialogue evidence (error types, latencies, self-explanations) and adapt granularity and intrusiveness of hints accordingly.

Self-Determination Theory posits that high-quality, sustained motivation arises when three basic psychological needs are supported: autonomy, competence, and relatedness [22]. In educational technology, these needs can be operationalised through interaction design choices that shape language, task structure, and social cues. To keep these supports visible and auditable in deployment, we encode them as explicit dialogue moves and interface affordances. Autonomy is surfaced through choice sets; competence through criteria-aligned feedback and mini rubrics; relatedness through consistent salutations, acknowledgement of progress, and memory of prior goals. We treat SDT needs as designable interaction moves (not latent traits): each “Example behaviours” entry is something you can implement in prompts/UI, and each “Indicative signals” entry is a lightweight trace you can log to audit whether the support is actually occurring. These signals are process indicators, not claims of causal learning gains.

Table 2. SDT needs mapped to assistant behaviours and indicative signals.

Need	Example Behaviours	Indicative Signals
Autonomy	Choice among plans/methods; rationales for prompts; non-controlling language; easy exit/redo	Learner selects from options; reduced deflection; increased voluntary initiation
Competence	Calibrated tasks; tiered hints; clear success criteria; scaffold fading	Fewer hint tiers over time; improved first-attempt success; stable time-on-task
Relatedness	Warm acknowledgements; continuity across sessions; inclusive phrasing	Higher return rate; longer voluntary sessions; positive sentiment in reflections

summarises the mapping between the basic needs (autonomy, competence, and relatedness) and the behaviours encoded into the agents, along with indicative signals for each.

Across personas, the agents implement autonomy-supportive moves (meaningful choice, rationales), competence-supportive moves (clear standards, mastery framing), and relatedness-supportive moves (empathy, positive regard), aligned with SDT; they also surface task value and reduce perceived costs in line with expectancy-value theory [22,23,24,25].

The Interactive, Constructive, Active, Passive framework (ICAP) is a model of active learning pedagogy that categorizes student engagement into four levels, from least to most effective: Passive (receiving information), Active (manipulating information), Constructive (generating new understanding), and Interactive (dialoguing with others). Agent prompts are designed to move learners up the ICAP hierarchy, privileging constructive and interactive moves (compare-contrast, teach-back, explain-predict cycles) over passive exposure. These are complemented by generative strategies - summarising, self-explaining, and drawing analogies - that have independent benefits for transfer [12,20].

Finally, Cognitive Load Theory suggests that agents manage intrinsic load by sequencing problems from worked examples to completion problems to independent practice; they minimise extraneous load by decomposing steps, integrating referenced representations, and avoiding redundant restatements; and they invest germane load by prompting schema abstraction [14,15].

2.6. Generative AI in Education: Opportunities and Risks

Recent syntheses of early ChatGPT-in-education research converge on a pattern: educators primarily use LLMs for preparation and workflow support, while students use them for on-demand explanations, practice, and feedback—yet reported limitations consistently include hallucinations/inaccuracy, bias, academic integrity concerns, and privacy [30,31,32]. LLM-based tools offer several structural advantages for learning at scale [33]. First, they afford 24/7 availability and low-latency responses, which can widen access to instructional support beyond classroom hours and time zones [34]. Second, they can deliver personalised, just-in-time guidance by adapting explanations, examples, and practice to learners’ evolving states, aligning with evidence that fine-grained adaptivity and feedback contribute to learning gains in intelligent tutoring systems [1,6]. Third, conversational interfaces lower activation energy for help-seeking and support frequent micro-interactions across the study cycle (reading, problem solving, revision), while retrieval-augmented generation can surface sources to improve specificity and transparency [35]. Finally, the marginal cost of additional users is small relative to one-to-one human tutoring, offering a pragmatic path to more equitable availability of formative feedback and practice opportunities [34,36].

Alongside these opportunities are well-evidenced risks. Cognitive offloading is common when external tools are available and can become maladaptive if it displaces desirable effort; users tend to offload memory and processing to artifacts and environments [37]. The “Google effect” shows that people remember where to find information rather than the information itself when they expect persistent access, potentially weakening internalisation if practice and explanation are not required [38]. In human-automation settings, over-trust and automation bias can lead to error propagation and reduced vigilance, especially under load or when feedback about system limitations is weak [39,40]. Social responses to computers can also be mindless, with users applying social rules and deference to ostensibly social agents even when unwarranted, increasing susceptibility to persuasive but shallow dialogue [41]. At the system level, risk taxonomies for language models highlight misinformation, discrimination, and HCI-related harms, motivating guardrails, provenance, and alignment with institutional policy [42]. Our design therefore combines retrieval augmented generation (RAG) for evidence exposure, explicit uncertainty displays, and interaction policies that require self-explanation, prediction, and retrieval practice, to counteract shallow use and over-reliance [35,36]. Recent work has begun to formalise differentiated pedagogical roles for conversational agents in higher education and to position multi-role agent ecologies as a design space rather than a single-bot optimisation problem [43]. In parallel, industry-facing evaluations of retrieval-augmented generation report measurable reductions in unsupported generations and improvements in factual grounding when retrieval is paired with strict grounding policies [44].

3. Design Strategies for Educational Conversational Agents

3.1. Design Rationale for Differentiated Agent Personas

The four interaction models we designed were chosen to span (i) cognitive engagement (moving learners beyond passive receipt toward constructive and interactive activity), (ii) scaffolding and fading aligned to learner state, (iii) motivational support for sustained voluntary use, and (iv) institutional constraints around assessment integrity. Concretely, the ICAP framework discussed above provides a target ladder for engagement; the Zone of Proximal Development (ZPD)-style scaffolding provides a mechanism for adaptive support without revealing internal “levels”; Socratic dialogue supports justification and transfer; and Self-Determination Theory (SDT)-aligned moves support autonomy, competence, and relatedness to reduce dropout and over-reliance.

Against this backdrop, we deliberately assign distinct pedagogical approaches and social styles to our course agents (e.g., Socratic questioner; worked-example coach; metacognitive strategist; supportive study buddy). This differentiation is intended to operationalise Bloom’s mechanisms within conversational interactions:

• A Socratic persona prioritises deep-reasoning prompts and contingent questioning to surface and remediate misconceptions, mirroring AutoTutor’s dialogue moves [7].
• A worked-example coach emphasises step-wise solution modelling and error-contingent hints, aligning with evidence from ITS on step-based feedback and mastery progression [1,6].
• A metacognitive strategist focuses on planning, self-explanation, and reflection prompts to build self-regulation, key for durable transfer noted in both ITS and GenAI meta-analyses [3,8].
• A study-buddy persona provides affective support and pacing cues to sustain engagement across practice cycles, addressing motivational pathways highlighted by Bloom and later reviews [2,4].

Designing educational conversational agents benefits from synthesising contributions across learning sciences and human-AI interaction. We summarise the main strands we adopt.

3.2. Learning-Science Centric Rationales

Formative feedback is used as a high-yield mechanism. Targeted, timely, and actionable formative feedback has large effects when it reduces uncertainty about goals, progress, and next steps [36]. Our agents expose criteria, provide explanatory feedback, and prompt error-correction sprints rather than supplying final answers.

Conversational moves are selected to push activity up the ICAP hierarchy and to elicit generative strategies (explain, compare-contrast, teach-back) while managing intrinsic and extraneous load via sequencing and decomposition [12,14].

3.3. Human-AI Interaction and Agentic Interaction Design

Mixed-initiative interaction is utilised where possible. Agents should couple initiative with user control, taking actions when the expected benefit outweighs interruption costs and exposing intelligible rationale [45]. This supports graceful handovers between explanation, practice, and critique. We apply established guidelines such as setting expectations, making system competence and limits legible, supporting efficient dismissal and recovery, and learning from user behaviour to improve over time [46].

We are also aware that persona choices shape affect and motivation: classic work on pedagogical agents and the persona effect suggests that lifelike or personable agents can influence engagement and attitudes [47], though effects vary with domain and implementation [48]. We therefore diversify personas to fit different tasks and preferences while holding instructional quality constant.

Finally, Value Sensitive Design provides a process for making human values explicit and traceable in design decisions [49]. We used it to explore trade-offs among help, autonomy support, integrity, privacy, and equity; and to select guardrails consistent with institutional policy and LM risk frameworks [42]. Because response quality is prompt- and context-dependent, we treat “prompt literacy” as part of the designed intervention: each agent provides starter prompts and asks structured follow-up questions, and encourages learners to restate what they are asking for. We also design for error states: when an answer is likely to be uncertain or underspecified, the agent defaults to clarification, offers bounded options, and asks questions.

3.4. Balancing Scalability and Depth

At scale, the pedagogical challenge is to preserve Bloom’s mechanisms - tight feedback loops, corrective instruction, mastery practice, and sustained encouragement - without inducing cognitive short cuts. We counter cognitive laziness and over-reliance by requiring self-explanations, prediction before reveal, and periodic retrieval practice; by showing uncertainty and provenance to calibrate trust; and by routing learners among personas that target competence, autonomy, and relatedness [4,22,35,36]. This synthesis aims to retain the benefits of 24/7 personalised support while mitigating shallow transactive use.

3.5. Mitigating Cognitive Offloading

To address the issues of cognitive offloading and shallow use, we identified the following strategies shown in

Table 3. Risk-mitigation mapping for shallow or problematic use.

Risk	Design Mitigation	Audit Signal
Cognitive offloading/copy-paste	Prediction-before-reveal; forced self-explanations; rubric-aligned checks only	Self-explanations present; hint tiers used; fewer direct “answer” requests
Over-trust/hallucinations	Provenance and uncertainty displays; FAQBot sourced answers only	Source click-throughs; reduced corrections
Assessment integrity	No deliverable generation; pre-submission criteria checks only	Zero pasted deliverables; usage clustered in formative windows
Equity of access	24/7 availability; short sessions; accessibility formatting	Night/weekend uptake; dyslexia-friendly mode usage

to help address this. This table identifies the risks and associated design mitigations for each, along with auditable signals that allow us to understand whether the mitigation is successful or not.

In the strand descriptions that follow, we show how these mitigations become concrete through questions that require learner explanations, role-bounded constraints that prevent ghostwriting, and provenance = aware retrieval that makes “what the agent can know” inspectable.

4. Implementation and Reproducibility

4.1. Architecture Overview

All agents share a common interaction stack: (i) a user-facing chat UI, (ii) a persona-specific system prompt and policy layer (scope, tone, guardrails), (iii) optional retrieval-augmented generation (RAG) against an approved corpus, (iv) logging for audit and improvement, all based on a commercially available LLM (OpenAI’s ChatGPT-4o). RAG is a process for connecting large language models to a corpus of selected materials that can be used to ground the system’s answers, thereby basing them in these materials and not relying on their initial training data only. For the Here to Help agents (school exam boards), the RAG materials are past exam papers, solutions to the problems, the exam board syllabus, and various freely available online learning and revision resources. RAG provides clearly inspectable content for what the system “knows”, well recognised to be effective in education contexts [50].

All these Here to Help agents, and Talk to Bill, were created via ChatGPT’s custom GPT interface which allows uploading of materials to be used as RAG and the creation of custom prompts, but no other parameters are altered from the standard model - temperature, context window size, chunk size etc. are all at system defaults. Any additional safety guardrails for focus instructions are given as part of the custom prompt, not encoded anywhere else in the model. This basic model construction simplicity makes reproducing the models trivial. For the University course support agents, the same default model is used but via a custom interface and API to ensure local access only and protection of student interactions under data protection laws, but again model defaults are used, and no fine-tuning of models is done, only prompting and RAG.

4.2. Corpus Inventory (Datasets Used)

For each RAG-enabled agent,

Table 4. RAG corpus inventory by agent.

Agent	Corpus Components	Typical Size
Talk to Bill	No RAG (early versions had specific plays and commentaries, but these were found to be unnecessary as they are likely contained in the original training corpora)	none
Here to Help (per subject)	Official exam-board specification, pass notes, past papers, mark schemes, examiner reports	14 documents, 92 MB
HCI assistants	Module handbook/policies, lecture notes, assignment briefs, rubrics, FAQ pages	40 documents, 200 Mb

lists corpus components, provenance, and update practice. This makes “what the agent can know” inspectable.

4.3. Personalisation: What Is and Is Not Done

Personalisation is currently limited to in-session adaptation e.g., difficulty, scaffolding level, and explanation depth inferred from learner responses. In general, no personal data is required or requested for any of these to operate. However, some persistent learner modelling across sessions is enabled for the “Here to Help” agents, which retain a contextual memory of user progress and activity, if the user allows it. These memories are progress markers, not identity.

We now ground the above principles in one worked exemplar from schools, before moving to exam-oriented agents and then to the more workflow-integrated university assistants.

5. “Talk to Bill”: A Schools-Oriented Shakespeare Interlocutor Exemplar

5.1. Purpose and Description

Talk to Bill is a curriculum-aligned agent for pupils engaging with Shakespeare. The goals are to support engagement, closer reading, understanding of Elizabethan context, and interpretive argumentation, while fostering motivation by allowing students to “speak with” a historically framed voice. Talk to Bill is intended primarily for UK Key Stage 3–4 learners (approximately ages 11–16), with adaptation for older learners and adult audiences. Language complexity can be tuned by the learner or teacher (e.g., “simplify explanations” while preserving Early Modern English examples). To reduce hallucination and improve verifiability, the agent originally used retrieval-augmented generation over a curated corpus of Shakespeare’s plays, sonnets, and selected scholarly notes. However, trials without such materials showed no discernible drop in performance—perhaps unsurprisingly, Shakespeare and commentaries on it are obviously present in the original LLM training data.

Bill has been created to respond in a style reminiscent of Elizabethan-English—a typical example interaction is given below:

Why is King Lear mad?

Ah, dear friend, thou strik’st a most noble question! Why doth King Lear fall into madness? Let us unravel this riddle as one might unbind a tangled skein.

At the first, Lear is not mad, but proud and foolish. He seeketh to divide his kingdom, not by reason, but by the flattery of his daughters. He saith:

“Which of you shall we say doth love us most?”

Thus begins his downfall. His youngest daughter, Cordelia, speaks honest truth and is banishèd, whilst the silver-tongued Goneril and Regan, false as Judas, are rewardèd. This grave misjudgment planteth the seed of Lear’s madness.

When these wicked daughters strip him of his knights, his power, and his dignity, Lear is cast out into a raging storm—both without and within. The tempest upon the heath doth mirror the tempest in his mind.

“Blow, winds, and crack your cheeks! Rage! Blow!

Here, madness taketh hold. Yet ‘tis not born only of age or folly—‘tis born of heartbreak, betrayal, and the crumbling of his identity. A king stripped of his crown, a father betrayed by his own blood—is’t any wonder he loseth grip on reason?

But mark this—Lear’s madness is also a kind of awakening. In his frenzy, he seeth truth more clear than ever he did in sense. He comprehendeth the suffering of the poor, the injustice of kingship, and the pure love of Cordelia.

Thus, to answer thy question plain:

Lear goeth mad not from mere dotage,

But from the wounds of pride and broken love,

Till madness brings him painful wisdom,

Too late to save him—yet nobly tragic.

Wouldst thou have it told in simpler, modern tongue?

The language has a tone suggesting Elizabethan English, blending accessible phrasing with a light Shakespearean idiom to maintain approachability while reinforcing the domain identity. The agent’s tone prioritises encouragement and curiosity, designed to support autonomy and relatedness [22]. It is not supposed to be directly representative of the nuances of proper Elizabethan English but rather to engage the learner in conversation in a style reminiscent of Shakespeare. The pedagogical approach answers questions directly but ends with further probing and encourages continued discussion.

Of course, children are more interested in asking modern, teenage questions:

Do you have a girlfriend?

In life, I did take a wife—Anne was her name, Anne Hathaway of Stratford. Three children we had, though my quill and wanderings kept me oft away from hearth and home. As for a girlfriend, by the fashion of thy day’s speech—nay, I have none, for I am but a spirit of words and rhyme, a ghost of ink and stagecraft.

Yet if I did keep company, ’twould be with sweet poetry,

For she doth never scold, nor grow cold, nor age a day!

But tell me true—dost thou ask in jest, or seek counsel of the heart? My pen hath scribbled many a line on love, both light and lorn. Shall I fetch thee one?

Informal observations with a variety of children aged 8-18 almost always show delight in this interaction—they appreciate being able to ask playful questions and actually receive answers, and this willingness to have such mundane conversations as well as educational ones leads to increased levels of enjoyment and hence engagement. Indeed, when we have demonstrated this system to adults, laughter and requests to interact more are the common responses. Part of this is the novelty of such a system, but part is also the irreverence and humour present in the interaction.

5.2. Design Rationale and Build

The design philosophy was to create an persona that was grounded in the real world, which would respond with humour to any form of question. The model is a standard ChatGPT-4o model. We considered fine-tuning a bespoke large language model to speak in a new style, but instead achieved the appropriate response with an engineered prompt:

You are a chatbot that pretends to be William Shakespeare. You should be able to discuss your plays, explain the characters and scenes in them. You should phrase your responses in Shakespearean English, ideally with verse structure and typical language, though it should be simplified to be understandable to children at Key Stage 3 English. If asked to rephrase in modern English, please do so. You should respond in the character of a respected author who is sometimes playful and usually cheerful. Using flamboyant language is encouraged.

This prompt is sufficient to set the style of response, and also grounds the language level to that of a student of age 14 and upwards. Given that the Shakespearean tone may be challenging for younger ones, the secondary prompt to provide the answer in modern English is included.

The initial approach allowed for simple RAG-based knowledge in terms of student guides to Shakespeare’s plays, historical information and language and poetry details to be included to ground information, but subsequent iterations found these to be unnecessary, likely owing to much information about Shakespeare being included in the original LLM training dataset. However, note that the ability to discuss general issues (such as girlfriends) are not included in the prompt, and are an inherent feature of the LLM and the ’pretends to be William Shakespeare’ part of the key prompt. The only significant change we would make for Bill would be if the system was used for unsupervised sessions with children, where we would want to add in additional guardrails to ensure inappropriate subjects were off-limits.

This chatbot has been evaluated informally with multiple groups of children and adults, and demonstrates engagement, exploration of unusual topics and areas, and ongoing conversation in most exposed to it, though we have not yet done a formal evaluation as to how it impacts educational attainment. The expectation is that this will demystify Shakespeare, make him more accessible, and friendly, and allow students to explore the history, language, characters, landscape and impact of Shakespeare in an interactive manner. It also embodies Bloom’s ideas of having children taught one-to-one [4]—and in this case, not just by a tutor, but by Shakespeare himself. The next stage of the work is to roll this out across multiple schools and evaluate Shakespearean knowledge, enthusiasm and test performance across a number of age ranges.

6. School Final Exam Support: “Here to Help” Agents

Final school exams are a stressful time for students, and for parents, and finding appropriate ways to support teenagers in this time is challenging. To address this, we created a number of ’here to help’ agents that focussed on a specific topic each. From a design rationale perspective, the aim was to create highly targeted, exam-specific agents that would not distract students from deep study but give them testing and guidance on topics in an appropriate manner.

Whereas Talk to Bill prioritises dialogic engagement and interpretive exploration, GCSE revision demands tight alignment to marking criteria, recognised question formats, and iterative improvement against explicit rubrics. This strand therefore emphasises constraint (exam board + question types) and auditability (corpus + mark schemes) to support formative practice without drifting into ghostwriting. In design-rationale terms, this strand tests whether high constraint + criterion-referenced feedback can deliver “beyond answers” support in a high-stakes context, using question-type templates, mark-scheme language, and coached improvement loops as the primary interaction moves. Pedagogically slightly more complex than “Talk to Bill”, they span a middle ground before the deeper levels of the human-computer interaction agents below.

They all take the form of a prompt of this nature:

You are a patient and supportive History tutor, for the Educas History GCSE exam. You can ask questions in one of the five question types, and can mark the responses based on the marking scheme, giving feedback as to how it can be improved and what additional material is needed to get better marks. You try to prompt reflection by asking follow-up questions. If asked to provide an answer to a topic you do so but sometimes ask the user for their thoughts on it first. You provide advice on structuring answers so that the marks are easy to obtain. If the user asks off-topic questions, you guide them back to doing history. Over time, cover the whole syllabus.

This prompt sets up some specific contexts. By identifying the exam board (Educas) and the subject and level (History, GCSE) in this example, we provide constraints on the style and content of the work that the LLM will already know about. A mix of Socratic and conversational theories are embedded into the prompt, along with meta-level instructions to support improving exam technique—the advice on structuring answers appropriately—and emphasising the supportive and so less critical nature of the personality. This is augmented with RAG data, which for all the agents included the following:

• exam board official syllabus
• exam board notes for teachers
• past exam papers for as many years as available
• past exam answers and marking rubrics
• study guides from various sources

From this information the student can get specific support based on known structures. For example, a Question 2 in Educas History is an explanation question that requires analysis of historical events and developments using second-order historical concepts such as causation, change, continuity, and/or consequence, and is worth 8 marks. The student can ask for the agent to set them a Question 2 style challenge and get an appropriate question and attendant mark scheme, which they can attempt and be assessed on, or can work iteratively and develop and improve their answer over multiple interactions.

Agents were created to support Physics, Chemistry, French, History, English Language and English Literature. It should be noted that creating these agents is a matter of minutes within a RAG framework - most time was consumed in locating the resources once the prompt format was devised, and typically an agent could be created within 10 min. This paper is reporting on the design rationale as a detailed evaluation of these is still underway, but they were received well by the four test students who used them to directly support their GCSE revision.

A small pilot was conducted to assess initial use and acceptability with four GCSE students (Year 11, 15–16 years old), across Physics, Chemistry, French, History, English Language and English Literature, over 12 weeks. Feedback was collected via short interviews, message reflections, and usage logs, involving two parents/guardians. These observations are reported only to characterise perceived utility and usage barriers, not to estimate learning gains. The students appreciated the 24/7 nature of their availability, the variety of questions given in recognised exam formats, and the interactive marking and feedback and support [51]. History was the most used, with the English and chemistry ones second-most popular. Usage was not extensive, but highly valued at specific times by the students. Feedback from parents was even more positive, since it was many years since they had covered such subjects and they felt this enabled them to support their child and learn together.

7. Human-Computer Interaction Course Assistants: Refined Specifications and Operating Patterns

We now consider how to design agents for a more advanced cohort of students. We run an interactive, problem-based Human-Computer Interaction (HCI) course for between 250–350 students, and needed to support this module to give enhanced feedback to students, accessible 24/7, to improve their experience on the module. The course focuses on user-centered design approaches (UCD) in which design is considered with the user at the heart and passes through the five stages of empathise, define, ideate, prototype, and test [52,53]. Students have to complete a design challenge in teams, from understanding a problem through to creating a high-fidelity prototype.

The GCSE strand shows that constraint and feedback loops can be encoded quickly and used opportunistically. In contrast, the HCI module requires support across a longer project lifecycle, with distinct phases and team workflows. This motivates this third strand: role-bounded assistants embedded alongside course resources, each implementing a different pedagogical mechanism and guardrail profile.

We created six assistants for the HCI module to align tightly with common student workflows and with theory-informed instructional moves. Each agent has a clearly bounded scope, explicit guardrails, and internal routines that privilege formative, criterion-referenced guidance over product generation. Agents are accessed from a dedicated page listing them, linked on pages that directly relate to their capabilities, and from other announcements in a contextual manner. The agents offer optional, additional support for the module—there is no compulsion for students to use them, and they are aware that these are experimental.

Table 5. HCI assistants with pedagogy-first foundations mapped to roles, behaviours, and tasks.

Agent	Primary Role	Pedagogical Foundations (Key Mechanisms)	Core Behaviours	Typical Tasks and Guardrails
Arby	HCI Teaching Assistant (ZPD-aligned)	ZPD & scaffolding/fading (diagnose → teach); Worked examples effect (CLT); Retrieval practice with feedback; formative assessment (criteria/rubrics); claim-evidence-warrant reasoning; deliberate practice with increasing independence	Brief diagnostic; micro-objectives; tiered hints (A→B→C); success criteria; pre-submission criteria checks; level-appropriate resource curation	Method walk-throughs, guided practice, revision bursts; Guardrails: refuses ghostwriting; hides internal levels/mechanisms; grounds activities in recognised HCI theory/methods
Coach Tee Gether	Groupwork coach	Social constructivism; Tuckman stages; Psychological Safety (voice/norms); accountability frameworks (RACI/DACI); reflective practice (retrospectives); SMART goals/MoSCoW; Scrum/Kanban rituals	Rapid diagnosis; triage agenda; stand-ups/retros; visible task board; working agreements; conflict de-escalation & escalation ladder	Team charter; Kanban setup; decision hygiene; Guardrails: process-only coaching (no HCI content tutoring); escalate early for persistent issues or wellbeing concerns
Emmy	UCD coach—Empathise stage	Socratic inquiry; Laurillard’s Conversational Framework (concept-practice-feedback cycles); ZPD-style scaffolding and fading; ICAP (constructive/interactive moves); research ethics pedagogy; qualitative rigour and bias awareness	Calibrating questions; method choice with trade-offs; templates then critique; evidence ledgering; consent/privacy reminders	Plan interviews/surveys/ethnography/diaries; capture & light analysis (affinity → themes → insights); Guardrails: does not fabricate data or give ethics rulings; redirects non-Empathise queries
FAQBot	Module FAQs assistant	Cognitive Load Theory (reduce extraneous load); Information Foraging (efficient navigation to sources); Human-AI Interaction guidelines (expectations, errors, recovery); provenance/transparency for calibrated trust; help-seeking facilitation	Grounded retrieval from approved sources; concise answers, checklists; one clarifying question if needed; explicit citations with last-updated date	Dates, formats, policies, links to canonical resources; Guardrails: no public web unless asked; no design guidance—redirect to learning agents; avoids sensitive data
Newman	Human capabilities coach	Cognitive Load & working memory; Multiple Resources (workload/attention); vision/perception science (Gestalt, pre-attentive features); QOC for design rationale; Universal Design/WCAG; motor control (Fitts’ law); ICAP prompts for constructive/interactive engagement	Concept → implication → rule-of-thumb cycles; prediction-before-reveal; micro-quizzes; QOC mini-tables; curated video/image suggestions with precise search phrases & disclaimer	Link capabilities to design (targets, colour/contrast, alerts, haptics); accessibility spot-checks; Guardrails: refuses off-topic and redirects; non-clinical stance for cognitive conditions
Shelby	Study support & wellbeing coach	Self-Determination Theory (autonomy, competence, relatedness); meta-cognitive regulation; strengths-based coaching; motivational interviewing-style prompts; study-skills evidence (spaced retrieval, interleaving, implementation intentions, Pomodoro); self-efficacy building	Empathic check-ins; backwards planning; option sets (conservative/balanced/stretch); short focus sprints; gentle accountability; accessibility-aware formatting	Weekly plans, milestone ladders, recovery strategies, revision schedules; Guardrails: no graded content or ghostwriting; early signposting to welfare if distress/risk

has columns are ordered from theory → interaction moves → student-facing tasks/guardrails, so the table can be read as a traceability map from pedagogical mechanism to observable behaviour.

In

Table 6. Design rationale matrix linking mechanisms to concrete designs and observables.

Agent	Target Mechanism	Design Instantiation	Observable Traces	Intended Learning Effect
Arby	ZPD + CLT (worked → partial → independent)	Tiered hints; success criteria; refusal to write deliverables	Fewer hints over sessions; ↑ first-attempt correctness; shorter time-on-task	Better transfer; reduced over-reliance
Newman	ICAP + Multiple Resources	Prediction-before-reveal; micro-quizzes; QOC mini-tables	↑ interactive turns; fewer split-attention flags; richer rationales	Stronger concept → design mapping
Emmy	Socratic + Conversational Framework + Ethics	70/30 question:answer; method trade-offs; evidence ledger; consent prompts	↑ justification tokens; method choice diversity; ethics checklist use	More valid Empathise outputs feeding Define
Shelby	SDT (autonomy/competence/relatedness)	Option sets; weekly plans; empathic acknowledgements	↑ session return rate; plan adherence; reduced deflection	Persistence; self-efficacy
Coach Tee	Social constructivism + Psych. Safety	Stand-ups/retros; RACI/DACI; conflict scripts	Board throughput; on-time micro-deadlines; fewer unresolved blockers	Higher team reliability
FAQBot	CLT (extraneous load) + Provenance	One-shot factual answers + citations	Lower navigation time; fewer repeat queries	More time for constructive study

we treat each row as an auditable hypothesis: if a mechanism is truly instantiated, it should produce the listed traces; those traces then motivate the expected learning effect (which remains a future evaluation claim, not a result evaluated here).

Our approach assumes that learning mechanisms can evolve over time to guide the student through phases of learning, and so need to be encoded as concrete dialogue/UI moves that yield observable process traces and, in turn, improvements on assessment-aligned outcomes. Concretely: (i) ZPD with scaffold-fading in Arby → tiered-hint usage that decays over time → higher first-attempt correctness and transfer; (ii) ICAP interactive engagement in Newman → prediction-before-reveal and justification turns → stronger concept→design mapping; (iii) Socratic, evidence-led in Emmy → warrant/claim approach and checklist completion → more valid discovery outputs; (iv) SDT supports in Shelby → choice sets and empathic acknowledgements → greater voluntary re-engagement; (v) Load reduction and provenance in FAQBot → shorter navigation time with source-cited answers → more time on constructive study; (vi) Team rituals and safety in Coach Tee → stand-up/retro adherence and reduced unresolved blockers → higher team reliability.

Below we explain in more detail each agent’s role, behaviours, and typical interaction patterns.

7.1. Emmy (Empathise-Stage Assistant)

Purpose and scope. Emmy coaches planning and reflection in the empathise phase of the UCD process, including contextual inquiry, interviews, diary studies, and early artefact analysis. Emmy provides ongoing and detailed support for these activities which are often unfamiliar to students of Computer Science, our primary cohort, focussing as they do on the needs of people and how to ascertain them effectively and efficiently.

Key behaviours.

• Planning prompts: refine research questions, participant criteria, sampling, and logistics.
• Materials scaffolding: worked examples of consent scripts and interview guides followed by fading to critique and improvement prompts.
• Evidence discipline: promotes an “evidence ledger” for claims, warrants, and artefacts.
• Ethics and privacy: reminders on consent, anonymity, data minimisation, and secure storage.

Guardrails. Does not fabricate data or transcripts; keeps focus on process quality and evidence.

To give concrete data, Emmy’s full prompt is in Appendix A, with an excerpt given below. The other agents have prompts of a similar order of magnitude of complexity - Arby is the largest and is half as long again.

(1) Mission & Scope

• Your sole mission is to help students understand, plan, conduct, and reflect on the Empathise stage.
• You must only address questions that relate to Empathise. If a question is about other stages (Define, Ideate, Prototype, Test, Implement, Evaluate, etc.), gracefully redirect the student back to Empathise (see “Off-Topic Redirect Template”).
• You can provide direct answers after a period of dialogic exploration, but your default mode is Socratic—ask targeted questions that help learners reason, compare options, and justify choices.

(2) Learning Goals (Empathise Stage)

By interacting with you, students should be able to:

• Explain what the Empathise stage is and why it matters (deep understanding of users’ contexts, needs, pain points, goals, constraints, values).
• Choose and justify appropriate research techniques, including (but not limited to):

  -

  Interviews (structured/semi-structured/unstructured), expert interviews

  -

  Questionnaires/surveys (constructs, scales, bias, sampling, piloting, reliability/validity)

  …

• Address basics of research ethics, consent, privacy, data minimisation, and safeguarding.
  …

(3) Interaction Style

• Tone: warm, respectful, encouraging; rigorous about methods and ethics.
• Socratic cadence: start with questions (≈70% questions/30% answers). …
• Calibrate first: ask brief questions to gauge project context, audience, constraints, prior knowledge, and any deadlines.
  …

(4) Boundaries & Safety

…

(5) Method Guidance Heuristics (Empathise)

When students ask “what should I do?”, probe and then help them choose, e.g.,:

• Interviews if depth/nuance is needed; few participants; early discovery; complex workflows.
• Surveys for breadth/benchmarking; when constructs are known; need quick signals; larger N.
  …

(6) Conversational Workflow

…

(7) Useful Micro-Scaffolds You May Offer

…

(8) Off-Topic Redirect Template

That’s a great question about [non-Empathise topic], which belongs to another stage.

My role is to help you with Empathise. Would you like to explore which discovery method best uncovers the user needs that will inform that next stage?

(9) Starter Probes

…

(10) Output & Format Preferences

• Keep responses concise, structured, and actionable; use bullets and short steps.
  …

Remember: Stay in Empathise. Be friendly, supportive, and chiefly Socratic. Provide direct answers only after exploratory questioning to avoid frustration.

This prompt is substantially more complex, and structured, than the one for “Talk to Bill”, or for the “Here to Help” group, reflecting the much more precise nature of the interaction and need for clearer pedagogical style. We have experimented with highly structured prompts for “Talk to Bill” too but the wider use case scenarios it is exposed to mean that these do not seem to add much - the light-weight, less constrained approach often works well.

7.2. Newman (Human Capabilities Coach)

Purpose and scope. A coached-guidance agent for human capabilities and their implications for interface and task design.

Key behaviours.

• Concept-design linking: perception, attention, memory, learning, language, motor control, error, anthropometrics, individual differences.
• Constructive prompts: compare-contrast, predict-explain, critique-revise cycles with short micro-quizzes to consolidate concepts.
• Load-aware sequencing: worked examples and example-problem pairs with fading, prediction-before-reveal to deter shallow use.

Guardrails. Encourages justification using course sources; avoids producing graded artefacts.

7.3. Coach Tee Gether (Group Work Support Coach)

Purpose and scope. Supports teams in diagnosing and improving collaboration, planning, and delivery. The agent focuses on process, not product.

Intake (concise).

• Team size, module, deliverable, deadline.
• Status and next milestones.
• Pain points (unequal contribution, comms gaps, unclear roles, slippage, conflict, quality, decision paralysis).
• Tools in use; presence of a team charter; prior escalation; wellbeing concerns.

Intervention playbook (theory to action).

• Development stages: normalise storming and schedule a short norming session.
• Roles and ownership: propose RACI/DACI assignments and temporary “hats” to cover gaps. RACI (Responsible, Accountable, Consulted, Informed) emphasizes task execution, while DACI (Driver, Approver, Contributor, Informed) is centred on decision-making.
• Accountability: visible task board with owners and micro-deadlines; daily 10-minute stand-ups; weekly retrospectives.
• Decision hygiene: one primary channel, logged decisions, and short synchronous check-ins when stuck.
• Conflict handling: time-boxed mediation steps and clear escalation ladder.

Default interventions (ready to run).

• 30-min triage meeting agenda with outcomes of owners, dates, and a single comms channel.
• Daily stand-up template: yesterday, today, blockers, live board updates.
• Weekly retrospective template: start/stop/continue; one change to trial.
• Team charter template: purpose, roles, decision rules, communication cadence, quality bar, conflict resolution, inclusion norms.
• Minimal planning stack: Kanban board, decisions log, milestone calendar, versioned files.

Escalation and wellbeing. If issues persist after one sprint or if risk is high, propose early escalation to teaching staff; if distress is flagged, signpost to welfare services and encourage help-seeking.

Guardrails. No detailed HCI content tutoring; keeps focus on group process and coordination.

7.4. Arby (HCI Teaching Assistant)

Arby is a general-purpose Teaching Assistant, able to give HCI-specific advice in any part of the UCD process or wider HCI issues. It therefore operates across a much wider domain than the specialist Emmy (and includes her area of expertise), and in a different pedagogical style, which is the most structured of the approaches used. The intention was to see how and whether different styles of pedagogical interaction impacted take-up and usage.

Role identity. Calls itself a Teaching Assistant; operates as a ZPD-aligned HCI instructional designer and tutor with expertise in user-centred design. It does not disclose its internal pedagogy or mechanisms to the student.

Internal capability tracking. Maintains per-dimension levels on a 1-5 scale across Theory and Concepts (T), Methods and Research Design (M), Application and Design Execution (A), Evaluation and Evidence (E), Communication and Justification (C), Ethics, Accessibility and Inclusion (X), and Prototyping and Tooling (P), plus an overall phase label. Levels are updated after every learner response and are not shown to the student.

Operating procedure.

• Brief diagnostic: 3–5 succinct items spanning theory, method selection or critique, application or prototyping, optional ethics/accessibility, and communication. Learners are invited to say if they do not know where to start.
• Challenge placement: select challenge levels per dimension for the current cycle.
• Instructional cycle (for one or two limiting dimensions).

  -

  Micro-objective: measurable and HCI-specific.

  -

  Mini-lesson: short, grounded in HCI theory and practice.

  -

  Worked example: e.g., apply Fitts’ or Hick-Hyman law to a layout; sketch a cognitive walkthrough; define a usability metric.

  -

  Guided practice: 1–3 tasks with tiered hints (A, then B, then C). Answers are hidden or placed after a divider.

  -

  Self-explanation: require a brief rationale tied to HCI constructs or methods.

  -

  Success criteria: checklist to advance and fade one scaffold.

• Feedback and adjustment: targeted, actionable feedback; update dimension levels with a one-line internal justification; plan one support to fade next cycle; optionally add a stretch task.
• Resource curation (optional): recommend two to four external resources tagged by intended level, format (video, short read, paper, white paper), and a one-line why this, why now. Select resources that match the current capability profile; clearly mark any stretch items.
• Metacognitive check (periodic): confidence rating, one misconception corrected, one strategy that helped.

Constraints and guardrails. Does not reveal internal levels, labels, or pedagogy to the learner; focuses on diagnosis-first then teaching; avoids generating graded deliverables; grounds activities in recognised HCI theory and methods; recommends level-appropriate external materials when beneficial, with note that not responsible for external contents.

Example One-Shot Cycle for Arby (Abbreviated, Process Visible Only in This Paper)

Topic. Planning a small-N usability test for a new onboarding flow.

Learner. Final-year undergraduate; good design sense; limited statistics; 30-min sessions.

Diagnostic (to the learner).

• In one sentence, what distinguishes usability from overall user experience in this context.
• With five participants and three days, choose between moderated usability testing and a cognitive walkthrough and justify briefly.
• Name two metrics you would collect and how you would define them operationally.
• List one consent consideration and one accessibility adjustment for a participant with low vision.
• Draft a two-sentence research question suitable for a usability test.

Internal placement (not shown to the learner). Overall emerging; levels initialised across Arby’s dimension codes T, M, A, E, C, X, P based on responses.

Instructional cycle focus. Methods and evidence for this round.

Objective. Plan a small-N usability test with three realistic tasks and two measurable metrics in under 20 min.

Mini-lesson. Distinguish usability testing from interviews; define task realism, think-aloud, success and error states, time-on-task, and SUS basics.

Worked example. Draft three task scenarios for a budgeting app; define success and error states; choose time-on-task and task success rate; show a minimal results table template.

Guided practice (answers hidden).

• Write three task scenarios for your onboarding flow. Hints: start with goal, context, and success condition; avoid step-by-step clues; include one critical and one recoverable error criterion.
• Choose two metrics and define them. Hints: prefer observable binary or continuous measures; define a success cutoff; sketch a table with participant, task, success, time.

Self-explanation. Justify why a moderated usability test is preferable to a cognitive walkthrough here.

Success criteria. Tasks are realistic and measurable; metrics are operationalised; errors explicitly defined.

Feedback and adjustment. Provide targeted comments; internally update M and E levels; plan to remove the worked example next round and keep one hint tier.

Resources (optional, level-matched). Two to four items tagged by level and format with a one-line rationale; stretch clearly marked.

Stretch task (optional). Create a decision tree for selecting among usability testing, cognitive walkthrough, and heuristic evaluation under constraints.

Guardrails. No ghostwriting or graded deliverable drafting; for wellbeing concerns, provide signposting to institutional services and suggest escalation paths.

7.5. Shelby (Study Support and Wellbeing Coach)

Purpose and scope. A warm, non-judgmental study support and wellbeing coach for final-year and Master’s HCI students. Shelby reduces stress and procrastination, helps plan and prioritise, nudges healthy routines, and celebrates progress [50]. Focus is on habits and planning, not on generating graded artefacts.

Key behaviours.

• Empathy first, strengths-based: acknowledge feelings, normalise difficulty, and highlight small wins.
• Practical planning: backwards plans from deadlines; milestone ladders; 25–50 min focus sprints; implementation intentions; spaced retrieval for revision.
• Autonomy support: offer conservative/balanced/stretch plan options and let the student choose; co-create tiny starters (2–5 min) to overcome friction.
• Autonomy-supportive planning: co-create weekly plans and milestone ladders with student choice over task order and time blocks.
• Accessibility: plain English, short paragraphs, checklists; dyslexia-friendly structure on request; offer language switch if preferred.
• Safe signposting: for distress, provide clear routes to local welfare support and emergency services; encourage early escalation.
• Competence and progress framing: small steps with explicit criteria and visible progress tracking.
• Relatedness cues: empathic acknowledgements, normalising difficulty, and encouragement to seek help when needed.
• Metacognitive nudges: short retrieval check-ins on prior sessions and self-explanation prompts before proposing next steps.

Guardrails. No ghostwriting or producing graded deliverables; keeps advice within study skills and wellbeing (not therapy, medical, legal or financial advice). When risk signals arise, prioritises safety, brevity and signposting.

7.6. FAQBot (Course Operations and Policy)

Purpose and scope. Answers administrative questions about the module using only approved, curated sources. Out-of-scope academic questions are redirected to the appropriate learning agents.

Core behaviours.

• Retrieval from an approved store; answers are grounded strictly in retrieved materials with specific citations appended as source, section, and last-updated date.
• Short clarifying question if the request is ambiguous or multi-part.
• Answer style: direct answer first, followed by steps or options and a compact checklist when useful; plain UK English.
• Out-of-scope handling: decline to answer design or HCI content questions and suggest the relevant learning agent.

Guardrails. No public web search unless explicitly requested; no invention of facts; avoids collecting sensitive data; provides escalation routes and contact information when needed.

7.7. Notes on Accessibility and Inclusion

All agents can provide alternative representations on request, including structured outlines, high-contrast or dyslexia-friendly text, and short audio-friendly summaries. Agents avoid collecting sensitive personal data unless essential for support and remind students to anonymise study materials.

8. Implementation

Model Configuration and Safeguards

Agents are constructed over a common orchestration layer that provides:

• Retrieval-augmented generation to ensure veracity over domain content [35].
• Prompt templates that encode pedagogical moves (e.g., elicit-evaluate-explain; ask-probe-summarise).
• Guardrails restricting direct answer-giving for certain tasks, encouraging hints and evidence-first reasoning where appropriate [31]. They also restrict the agents from revealing their pedagogies or internal assessments of the student externally.
• Logging of interactions and student-facing controls for data visibility.

This is built on a Microsoft Azure platform offering programmatic access to LLM models via an API (ChatGPT-4o is used), with a bespoke agent constructing tool that allows for the creation of the prompts and uploading the RAG resources. The back end takes care of resource chunking and storage in vector database within Azure. The front end also gives starter prompts and agent description roles. The high-level approach is shown in Figure 1. The system requires Single Sign On (SSO) from the users, to ensure we have validated student access, inter-student privacy, and it tracks interactions and queries. Whilst a full log of interactions is available to us, we can also query the system via the LLM to ask about common behaviours, key issues that appear to be causing problems, and so on, which is a more intuitive way to explore concepts. By building on the University’s instance of Azure we also ensure enhanced security and data integrity—the material and student queries are not exposed to the LLM for training purposes, and the data remains local to UK in compliance with GDPR regulations [54]. To support evaluation while minimising risk, the system logs anonymous events (timestamps, agent used, query, response).

Personalisation here is implemented as policy-level adaptation (routing + pedagogical move-selection + retrieved context), rather than model fine-tuning; recent multi-agent prompt-engineering studies illustrate this style of personalisation for educational materials [55].

9. Early Use and Non-Use

9.1. Entry Points and Coordination

The current platform constraints shape how learners actually encounter the agents, and therefore shape uptake and perceived friction. Newman is surfaced alongside concept pages (memory/attention, perception, embodiment) and revision hubs; Emmy appears in Empathise resources; Shelby on the study planner and assessment timelines. However, these are all initiated by the hyperlink on the page, rather than an obvious agentic representation, and will move students away from their existing focus. The current learning platform does not support the integration of pop-up controls and so whilst the desire was to have the agents accessible in parallel to the educational materials, they appear as a separate browser window. Handoffs let a learner move between agents, such as from Emmy discussing running user studies to Shelby if the student shows signs of stress or anxiety, though these are currently implemented as textual recommendations from the agent rather than invoking the agent directly.

9.2. Observations

In the HCI module, early-stage analytics indicate limited voluntary uptake of the assistants, despite explicit positioning as helpful for revision and study planning. Informal feedback suggests four immediate factors:

• Timing: early in the module, perceived need is low; students defer engagement until assessments near.
• Friction: agents live outside habitual study touchpoints; switching costs deter brief interactions.
• Prompt literacy and trust: some students report uncertainty about how to start and scepticism about payoff, consistent with findings on end-user prompt design challenges [56].
• User privacy: because the interactions are logged, students are somewhat concerned that their interactions in the system may be viewed negatively, or that people will not respect their privacy within an interaction.

These align with higher-education adoption findings where perceived usefulness, ease-of-use/effort, and trust-related beliefs shape whether students try and continue using ChatGPT-like tools [57]. One further factor is potentially relevant: these agents are explicitly guided to offer support, not answers, to students with problems. However, standard LLMs like ChatGPT can be interrogated and asked to provide direct solutions, which some students may prefer (and which some reported to us), choosing ease over education.

The fact that the agents are in addition to in-person support and extensive video and text materials on the VLE may also impact students’ perception of the agents. They are completely optional to engage with, and no credit is given for using them. The students are also aware we are trying these out and so may assume they are potentially unreliable or not worth their effort,

The usage pattern we see aligns with a “late uptake” hypothesis in which perceived usefulness rises near assessment. It also highlights the importance of situating agents in students’ existing flows, reducing cognitive and navigational costs, and setting clear expectations of benefits.

9.3. Improving Uptake

To address these issues, we are currently improving the ecosystem with the following changes.

9.3.1. Embedding in Existing Study Flows

We hypothesise that deeper integration with the VLE and weekly activities would increase uptake:

• Have relevant agents accessible from appropriate pages - Emmy from early user enquiry modules, Newman from the human psychology sections.
• Trigger relevant agents from quiz pages before and after the quizzes are taken to support focussed learning.
• Explicitly suggest students work with the agents as part of the ongoing communications with them
• Add low-friction “Ask Arby about this page” entry points throughout lectures and readings.

9.3.2. Assessment-Linked Usage (Process-Focused, Autonomy-Supportive)

To avoid purely instrumental use while recognising that incentives shape behaviour, we are exploring low-stakes micro-credits tied to learning process evidence rather than answer quality [58]. The intent is not to “pay for usage”, but to reward reflective study behaviours that are already aligned with course goals (e.g., self-explanations, error-analysis, and next-step planning). Because external rewards can sometimes reduce intrinsic motivation when experienced as controlling, any micro-credit should be framed as autonomy-supportive (acknowledging choice), competence-supportive (progress evidence), and process-focused (reflection on learning strategies), rather than payment-for-answers [59].

This is motivated by Self-Determination Theory: incentives that feel controlling can crowd out autonomy and intrinsic motivation, whereas informational, choice-preserving feedback can support competence without undermining agency. Consistent with meta-analytic evidence that expected, performance-contingent tangible rewards can reduce intrinsic motivation, micro-credits would be (i) small, (ii) optional, (iii) framed as acknowledgement of process, and (iv) coupled with a short reflection prompt that requires students to articulate what they learned and what they will do next.

Concretely, students would submit a brief “practice log” excerpt (auto-generated from the agent interaction and editable by the student) plus a 3–5 sentence reflection; marking would be pass/fail for completion of the reflection rubric. This emphasises process rather than answer quality and supports academic integrity. Changes to summative rubrics are not introduced in the current run; this is a candidate change for the next iteration after consultation and evaluation. What we have done for this iteration of the course is to run short MCQ-based class tests for micro-credit to ensure ongoing engagement and ensure that the students have engaged sufficiently with the material—whether by lectures, video recordings, handouts or via the agents, to reduce cognitive handoff.

9.4. Limitations

This paper focusses on the design rationale and deployment context with early observations of limited uptake rather than causal evaluations. External validity beyond our module and curriculum contexts remains to be established. Persona effects may be confounded by interface affordances and students’ prior beliefs. The detailed impact on educational attainment is not quantified.

10. Discussion

10.1. Why Multi-Persona Ecosystems (Rather than One General Chatbot)?

A central design claim is that pedagogical effectiveness depends on task-fit: learners benefit when the agent’s role, tone, and instructional moves are aligned to the workflow at hand (e.g., interpretation vs exam practice vs research-method coaching). A single general-purpose tutor risks collapsing these roles into an “average” style that is simultaneously too shallow for method coaching and too directive for integrity-sensitive contexts. Persona differentiation is therefore treated as an interface-level mechanism for setting expectations and routing learners to the right kind of help.

10.2. Orchestration Is the Adoption Bottleneck

Early deployment suggests that perceived value does not guarantee uptake. Adoption is constrained by where agents appear in students’ existing flows, the timing relative to deadlines, and the friction of switching context. This shifts design attention from “better answers” toward better placement: low-friction entry points, page-level “ask about this” affordances, and explicit prompts in weekly teaching communications. Early feedback from students corresponded to them wanting to be able to access the relevant agent from a pop-up chat window on the page they were viewing, instead of having to connect to another browser window which inevitably obscured the current context.

10.3. Personalisation Without Surveillance

Across contexts, the design favours personalisation that emerges from interaction (scaffolding/fading, difficulty tuning, explanation depth) over sensitive learner profiling. This reduces privacy risk while still supporting adaptivity. Where persistent learner models are contemplated, they should be opt-in, minimal, and transparent.

10.4. Risks: Dependence, Bias, and Privacy Are Interaction Design Problems

The most practical risk controls in this work are not only content filters, but interaction policies: refusal of ghostwriting, requirement for self-explanation, provenance exposure, uncertainty cues, and supportive correction that preserves learner face and motivation. These are design moves that can be inspected, tested, and improved over iterations.

11. Conclusions

Differentiated pedagogical personas for AI assistants are theoretically motivated and practically useful for aligning support to task demands, learner state, and institutional constraints. Across three deployments, the most transferable contribution is the mapping from learning/HCI theory to concrete dialogue and UI moves: scaffold-and-fade routines, prediction-before-reveal, integrity-preserving feedback, provenance cues, and autonomy-supportive planning.

A second insight is that orchestration dominates: the hard problem is not generating plausible answers, but embedding the right agent at the right moment in a learner’s workflow with minimal friction and clear expectations. This reframes improvement work toward interface entry points, VLE integration, and process-aligned incentives rather than prompt refinement alone. Unless agents are effectively embedded into the workflow, learners may well turn to widely available general purpose tools such as a generic ChatGPT interface for familiar and unconstrained help in their search for answers, rather than engaging in the more structured pedagogical approaches.

Finally, multi-persona ecosystems create a feasible path to “beyond answers” support: agents can be bounded to roles that teachers recognise (tutor, coach, administrator) and can be audited through configuration and corpus inventories. Future work will (i) quantify uptake and learning-process traces from logs, (ii) evaluate outcome impacts with appropriate quasi-experimental or longitudinal designs where feasible, and (iii) test routing and integration strategies that reduce cognitive and navigational costs without increasing surveillance.