Toward a Digital Emotional-Intelligence Tool for Faculty Motivation: A Self-Determination Theory-Grounded Feasibility Study of Theory-Guided Message Personalization

Abstract

Most computational work on emotional intelligence in education targets students; where the emotional states of teaching staff are modeled at all, the signal is typically a static questionnaire score rather than an input to an automated response. This study asks a narrower question: does routing a detected emotion through Self-Determination Theory—rather than naming the emotion, or using generic language—produce a message that university faculty themselves rate as better, and does that advantage survive once the underlying emotion classifier is realistically imperfect? Using two public emotion corpora (GoEmotions and ISEAR), a 75-item researcher-authored and independently unvalidated set of faculty-oriented stress vignettes, four classifier families spanning lexicon counting to large-language-model few-shot prompting, a four-level message-personalization ablation (generic, emotion-only, need-only, full situational context), and blind ratings from 30 higher-education faculty and researchers, two findings are reported. First, when classifiers are compared on a matched label space, the few-shot large-language-model classifier performs best on both out-of-domain evaluations without fine-tuning, including 0.893 accuracy on the synthetic faculty-oriented vignette set. Second, the human evaluation does not support a uniform monotonic benefit from progressively adding personalization. Need-only routing does not yield a consistent gain over emotion-only messages, whereas the full-context condition shows its clearest positive signal for relevance and motivational usefulness; these two contrasts are significant in the crossed mixed-effects analysis but are not uniformly significant under the rater-level paired tests. The automatic text-similarity proxy does not detect this contextualized-condition advantage. Because the faculty vignettes are synthetic and were not independently content-validated, the results should be interpreted as evidence of technical and methodological feasibility rather than ecological validity or deployment effectiveness.

1. Introduction

Faculty burnout is not a peripheral wellbeing concern; it is an institutional-performance problem. Systematic reviews link sustained occupational stress in university teaching staff to emotional exhaustion, depersonalization, and declining engagement with teaching and research [1]. Universities now face overlapping pressures—growing administrative workload, more diverse and demanding student populations, and continuous digital transformation of teaching infrastructure—that leave little institutional capacity to act on individual faculty distress as it occurs. Existing faculty-wellbeing interventions are heterogeneous and are typically delivered as human-led, organizational, or structured programmatic support rather than as immediate responses to an individual’s text-expressed situation [2,3]. A tool able to route an everyday, text-based signal of distress to something more specific than a form letter would be a useful addition to that toolkit, provided it can be shown to actually help rather than merely sound personalized.

Several adjacent literatures bear on this idea, but they address different parts of the problem. A systematic review of 146 studies of artificial intelligence in higher education found limited educator involvement and a weak connection between many AI applications and pedagogical theory [4]. A separate review of post-secondary academics’ coping and emotion-regulation research documents the importance of faculty stress and emotion for wellbeing and performance, while describing interventions primarily in human and organizational terms rather than as text-driven computational support [5]. Affective-computing and AI-in-education research itself remains predominantly learner-facing: tutoring systems and learning-analytics platforms detect frustration, confusion, or disengagement in students, not in the staff who teach them. Where computational methods have been applied to teacher or faculty emotion, the work has largely been descriptive. Chen et al. [6] analyzed roughly one million K-12 teacher forum posts with a word-frequency lexicon to characterize teacher emotion over time, without producing a downstream response. Li [7] applied a deep-learning sentiment model to student evaluation-of-teaching text, where the modeled emotion belongs to the student rather than the instructor. These strands motivate a narrower question about whether an explicitly theory-routed, faculty-facing response pipeline adds measurable value.

Much of the organizational-behavior literature operationalizes emotional intelligence as a relatively stable individual-difference construct, using instruments such as the Wong and Law Emotional Intelligence Scale [8] or the Trait Emotional Intelligence Questionnaire [9]. Such work is informative about individual EI differences and their associations with outcomes, but it does not directly test what an institution should do, in the moment, in response to a faculty member’s expressed emotion. The present study is concerned with that second, narrower question.

We do not claim to have built an emotionally intelligent assistant, and we are skeptical of work in this space that makes that claim without testing what, specifically, the “emotional intelligence” is doing. Instead, we ask a question that can be answered with evidence we can actually collect: in a faculty-support pipeline that detects an emotion from short first-person text and then generates a motivational message, does routing that emotion through a recognized motivational theory—Self-Determination Theory (SDT) [10]—produce a message that faculty themselves judge to be better than one that merely names the emotion or says nothing in particular? And does that advantage survive once the emotion classifier driving the pipeline is realistically imperfect, rather than a gold label?

We answer this with three linked experiments built on public emotion corpora, a researcher-authored synthetic scenario set, and blind human judgment, with no naturally occurring faculty emotional data collected at any stage. Experiment 1 benchmarks four families of emotion classifier—lexicon counting, supervised TF-IDF, a fine-tuned transformer, and large-language-model few-shot prompting—on two public emotion corpora and measures how each degrades under domain shift. Experiment 2 applies the same classifiers to 75 faculty-oriented synthetic stress vignettes. Experiment 3 decomposes “personalization” into four nested levels—generic, emotion-acknowledging, SDT-need-routed, and fully situation-grounded—and has 30 higher-education faculty and researchers rate the resulting messages blind to which level produced them. After auditing the rating data and re-running the full analysis, the evidence is more qualified than in the initial submission: SDT-need-routing alone does not produce a consistent human-rated gain, while the full-context condition shows selective positive effects, strongest for relevance and motivational usefulness. These effects are supported by the crossed mixed-effects model but are not uniformly significant under the primary rater-level paired tests. The automatic text-similarity proxy does not capture this pattern.

The novelty of the study lies not in proposing a new emotion classifier, but in making theory-routing an explicit and ablatable design component between emotion recognition and message generation. The paper makes four contributions: (1) it synthesizes a specific, underserved intersection between faculty emotion research, AI in higher education, motivational theory, and generated support; (2) it provides a label-space-fair benchmarking procedure across four classifier families, including an explicit out-of-label-space diagnostic that shows how an unmatched comparison can reverse a methodological conclusion; (3) it introduces a four-level personalization ablation that separates emotion acknowledgement, SDT-need routing, and full contextualization rather than treating “personalization” as a single condition; and (4) it reports a blind multi-rater evaluation, including rater-level, crossed mixed-effects, and formal inter-rater reliability analyses, showing that an automatic lexical proxy and human judgment emphasize different properties of supportive messages.

2. Theoretical Background

This section synthesizes the theoretical and empirical literature that motivates the artifact and its evaluation design. The literature is organized from the problem context to the proposed mechanism: faculty stress and existing support, emotional and affective AI, Self-Determination Theory (SDT) as a model of work motivation, and the technical problem of emotion recognition under domain shift. The final subsection integrates these streams, compares representative studies closest to the present problem, and derives the five research questions evaluated in the three experiments.

2.1. Faculty Stress, Emotion, and Support in Higher Education

The need for faculty-focused support is well established, although the evidence base is distributed across occupational-health, higher-education, and emotion-regulation research. Watts and Robertson [1] identified burnout as a recurring problem among university teaching staff, with emotional exhaustion and reduced professional functioning among the central outcomes reported in the literature. Urbina-Garcia’s systematic review of 28 studies [11] similarly concluded that university work environments are associated with high levels of stress and burnout and low levels of wellbeing. Salimzadeh et al. [5] focused more specifically on how post-secondary academics cope with stress and regulate emotion, showing that academic work involves recurrent affective demands and that coping strategies are shaped by both individual and organizational conditions. Together, these reviews establish faculty emotion as an occupationally meaningful phenomenon rather than a peripheral individual trait.

The intervention literature also shows that faculty wellbeing can be improved, but it does not yet converge on a single mechanism or delivery model. A systematic review and meta-analysis of university-faculty mental-health interventions identified 10 eligible publications with 891 participants and found substantial heterogeneity in intervention effects [2]. A later review of 21 studies published from 2020 to 2024 identified social support, training, supportive work environments, psychological-capital approaches, and other structured programs among the interventions used to reduce lecturer burnout [3]. Recent higher-education studies have also begun to connect AI use with faculty wellbeing: Cambra-Fierro et al. [12] modeled ChatGPT adoption and faculty happiness, energy, and stress, while Huang and Zhao [13] used SDT to examine how AI literacy relates to faculty work–life balance and job satisfaction through autonomy, competence, and relatedness. These studies bring AI and faculty wellbeing closer together conceptually, but they remain survey-based explanatory models rather than evaluations of an emotion-responsive support artifact. The intervention reviews likewise summarize predominantly human-led, programmatic, or organizational approaches. None of these evidence streams, by themselves, establishes whether a just-in-time text pipeline can take a faculty member’s own description of a work situation, infer an affective state, and produce a theory-guided supportive response. The present study addresses that narrower design problem; it is not intended as a substitute for organizational change, counseling, or clinical care.

2.2. Emotional Intelligence, Affective AI, and Generated Support

Emotional intelligence (EI) is commonly framed as a set of capacities for perceiving, using, understanding, and managing emotion [14], while organizational and educational studies also operationalize EI through established ability- or trait-oriented instruments such as WLEIS and TEIQue [8,9]. The present artifact does not estimate a faculty member’s stable EI and does not claim that the software itself possesses human emotional intelligence. Instead, “digital emotional intelligence” is used in a deliberately functional sense: the system detects an expressed affective signal, relates that signal to a motivational construct, and generates a response whose usefulness can be evaluated. This operational boundary separates the present design problem from psychometric EI assessment and makes each computational stage independently testable.

Educational AI has developed rapidly, but the dominant unit of analysis remains the learner. Zawacki-Richter et al. [4], in a systematic review of 146 higher-education AI studies, reported limited educator involvement and weak pedagogical grounding in much of the field. More recently, Vistorte et al. [15] reviewed 41 studies using AI to assess emotions in educational settings; the reviewed applications primarily captured students’ emotional states during learning and emphasized challenges involving accuracy, privacy, and cross-cultural validity. This learner-centered pattern is important because it means that advances in educational affective computing do not automatically establish the validity of a faculty-facing support system.

The comparatively small body of computational work involving teachers also stops short of the full intervention loop examined here. Chen et al. [6] used computational text analysis to characterize emotion in approximately one million K–12 teacher forum posts, but the detected emotions were descriptive outputs rather than inputs to a support response. Li [7] applied deep-learning sentiment analysis to student evaluations of teaching, where the modeled affect belongs primarily to students rather than faculty. Thus, these studies demonstrate that educational text can carry useful affective information, but not that detecting such information improves a response delivered to the faculty member who expressed it.

A second relevant stream comes from conversational and emotional-support systems outside higher education. Li et al. [16] synthesized 35 studies of AI-based conversational agents for mental health and wellbeing and found that user experience depends strongly on communication quality and the human–AI relationship. Sanjeewa et al. [17] reviewed 19 empathic conversational-agent studies and found that human evaluation was common (16 of 19 studies), while direct evaluation of empathic features was much less consistent. Sharma et al. [18] showed in a randomized trial with 300 peer supporters that AI-in-the-loop feedback can increase empathy in real text-based supportive conversations. At the language-generation level, Zheng et al. [19] demonstrated that LLM-generated self-chat data can improve a smaller emotional-support chatbot, while ESC-Eval [20] showed that evaluating LLM-based emotional-support conversations requires dedicated human annotations and that specialized systems still trail human performance. These studies establish the feasibility of generated or AI-assisted support, but they concern mental-health, peer-support, or general dialogue settings rather than occupational support for university faculty.

This literature also motivates the evaluation strategy used in the present study. Person-centered communication research shows that more person-centered supportive messages are generally perceived as more effective [21]. At the same time, NLG evaluation research warns that automatic metrics often capture surface similarity rather than the properties people actually value [22]; perceived empathy in dialogue systems also requires explicit human measurement rather than being inferred from wording alone [23]. Therefore, an automatic lexical score can be useful as a reproducible proxy, but it cannot by itself establish that one supportive message is better than another.

2.3. Self-Determination Theory and Faculty Motivation

Self-Determination Theory [10] proposes that high-quality motivation and psychological functioning depend in part on satisfaction of three basic psychological needs: autonomy, competence, and relatedness. The Work Tasks Motivation Scale for Teachers (WTMST) [24] applies this framework to teaching work and distinguishes autonomous from controlled motivation across concrete work tasks. Importantly for the present study, SDT has also been examined directly in higher education. Stupnisky et al. [25] tested an SDT-based model with 1671 faculty members from 19 universities and found that autonomy, competence, and relatedness positively predicted autonomous motivation, which in turn predicted greater use of effective teaching practices.

Evidence across the broader teacher literature points in the same direction. Slemp et al. [26] synthesized 1117 correlations from 102 independent samples and found that autonomous teacher motivation is associated with wellbeing, job satisfaction, and autonomy-supportive functioning, whereas controlled motivation is more closely associated with distress and burnout. A recent systematic review of 102 studies of teachers’ basic psychological needs likewise found that need satisfaction is repeatedly associated with motivation, wellbeing, and professional resilience, while need frustration is linked to adverse outcomes including burnout [27]. In directly adjacent higher-education applications, Huang and Zhao [13] found that autonomy, competence, and relatedness mediate relationships between AI literacy and faculty wellbeing outcomes, and a separate survey of 482 university faculty [28] explicitly applies SDT to explain how institutional digital transformation translates into faculty performance, with digital self-efficacy as the mediating psychological mechanism. This recent work is particularly relevant because it confirms that SDT can organize questions about faculty experience in AI- and digitally-intensive academic settings from more than one independent research group; however, neither study tests emotion detection, theory-guided message generation, or immediate supportive communication. These results provide a substantially stronger basis for using autonomy, competence, and relatedness as organizing constructs than would a purely intuitive personalization scheme.

However, this evidence does not validate the specific emotion-to-need mapping used in the present artifact. SDT describes motivational needs and the conditions that support or frustrate them; it does not imply that a discrete emotion deterministically identifies one need. We therefore use emotion-to-need routing as an explicit design hypothesis that can be ablated and tested, not as a validated diagnostic instrument. This distinction is central to the study: a useful theoretical scaffold can motivate a response architecture even if a particular operationalization of that scaffold fails to improve message quality.

2.4. Computational Emotion Recognition and Domain Shift

Text-based emotion recognition provides the upstream signal for the proposed artifact. Many NLP datasets ultimately relate their labels to compact emotion taxonomies such as Ekman’s basic emotions [29]. GoEmotions [30] provides 58,000 Reddit comments annotated for 27 fine-grained emotions and a mapping to broader emotion groups, whereas ISEAR [31] contains approximately 7600 self-reported emotional situations collected across multiple countries. These corpora differ not only in size but also in genre, elicitation procedure, and label inventory.

Such differences matter because NLP models routinely lose accuracy when training and target distributions differ [32]. Textual emotion classification is not an exception: recent cross-domain work explicitly treats domain adaptation as a central challenge for emotion models [33]. For a faculty-facing system, this is unavoidable because large public emotion corpora are not drawn from naturally occurring university-faculty work reports. A classifier can therefore look strong in-domain and still fail on short occupational narratives. Cross-dataset evaluation introduces a second problem: if a source model can predict labels that do not exist in the target benchmark, apparent accuracy differences can partly reflect label-space mismatch rather than genuine robustness. The first two experiments consequently evaluate both domain shift and label-space alignment before the classifier is used as an upstream component of message generation.

2.5. Research Synthesis, Comparative Literature, and Conceptual Framework

The literature supports each component of the proposed artifact separately, but the components are usually studied in different research traditions. Faculty-wellbeing studies establish the problem and evaluate mainly organizational or human-delivered interventions [2,3,11]; recent faculty–AI studies examine adoption or AI literacy as predictors of wellbeing [12,13]; educational affective-computing studies develop methods for detecting emotion but remain mainly learner-facing [4,15]; SDT research provides strong evidence that autonomy, competence, and relatedness are relevant to teacher and faculty motivation, including in digitally-intensive settings [25,26,27,28]; and emotional-support dialogue research shows that generated supportive language can be evaluated meaningfully only with human judgment in addition to automatic metrics [17,18,20]. Table 1 makes this separation explicit for representative studies closest to the present problem.

Table 1. Comparison of representative literature streams most directly related to the present study. “Varies” indicates a review covering heterogeneous systems rather than a single fixed architecture.

Study	Population/Setting	Automated Emotion Inference	Explicit Theory/Framework	Generated/Adaptive Support	Human Evaluation of Support
Watts and Robertson [1]	University teaching staff	No	No	No	No
Urbina-Garcia [11]	University academics	No	No	No	No
Stupnisky et al. [25]	University faculty	No	SDT	No	No
Cambra-Fierro et al. [12]	University faculty/ChatGPT adoption	No	TAM	No	No
Huang and Zhao [13]	University faculty/AI literacy and wellbeing	No	SDT	No	No
Chen et al. [6]	K–12 teachers/online forums	Yes	No	No	No
Zawacki-Richter et al. [4]	Higher education AI review	Varies	Varies	Varies	Varies
Vistorte et al. [15]	Educational emotion-assessment studies (mainly learners)	Yes/varies	No	Varies	Varies
Li et al. [16]	Mental-health and wellbeing conversational agents	Varies	Varies	Yes	Yes
Sanjeewa et al. [17]	Empathic mental-health conversational agents	Yes/varies	No SDT	Yes	Yes
Sharma et al. [18]	Peer-to-peer mental-health support	Not the primary task	No SDT	AI-assisted	Yes (RCT)
Zheng et al. [19]	Emotional-support dialogue	Not the primary task	No SDT	Yes	Not the primary evaluation
Present study	Higher-education faculty	Yes, text-based	SDT routing	Yes, four-level ablation	Yes, blind expert ratings

The comparison does not support an absolute claim that the individual components are new. Rather, it identifies a specific unresolved intersection: whether a faculty-facing text pipeline gains measurable value when detected emotion is routed through an explicit motivational theory before support is generated, whether additional situational context matters beyond the theory label, and whether those conclusions remain stable when the emotion classifier is imperfect. This gap is important for two reasons. First, theory-guided personalization should not be assumed beneficial merely because its output sounds tailored; the contribution of each routing stage must be isolated. Second, a pipeline intended to work outside its training domain must be evaluated under the classifier errors that real deployment would introduce.

Accordingly, the conceptual framework treats personalization level as the principal downstream design factor and classifier quality as an upstream robustness factor. The four personalization levels incrementally expose the response mechanism to no individualized signal (generic), detected affect (emotion-only), an SDT-derived need (need-only), and the need plus the original situation (full context). The evaluated human outcomes are relevance, specificity, SDT alignment, and motivational usefulness. Figure 1 shows how these components map onto the three experiments.

Figure 1. Conceptual framework and study workflow. Experiment 1 evaluates emotion-classifier robustness under domain shift using the public corpora; Experiment 2 evaluates the same classifier families on the researcher-authored synthetic faculty-oriented vignette set; Experiment 3 evaluates four progressively enriched message conditions using an automatic lexical proxy and blind human ratings. The emotion-to-SDT-need mapping and message-generation stages are evaluated in Experiment 3 rather than in Experiments 1–2.

The literature synthesis leads to five explicit research questions:

RQ1. 

How does classifier choice affect robustness under domain shift when emotion label spaces are aligned, and to what extent can out-of-label-space predictions distort that comparison?

RQ2. 

Which emotion categories remain difficult to classify in the researcher-authored faculty-oriented synthetic vignette set, and how does this vary across classifier families?

RQ3. 

On a reproducible automatic lexical proxy, does progressively adding emotion information, SDT-need routing, and full situational context improve generated- message quality?

RQ4. 

In blind expert judgment, does SDT-need routing improve message quality beyond emotion acknowledgement alone, and does full situational context provide a further benefit after rater and item variation are accounted for?

RQ5. 

Do the automatic proxy and blind human evaluation agree about the relative value of the four message-personalization conditions?

3. Materials and Methods

This section operationalizes the conceptual framework as a design-science artifact and describes the datasets, classifiers, synthetic faculty vignettes, SDT mapping, message-generation conditions, evaluation protocol, and statistical analyses used to assess it.

3.1. Design

This study follows design science research (DSR) principles [34,35]: it builds and evaluates an artifact rather than treating the artifact itself as a validated psychological instrument. The DSR stages are mapped explicitly as follows. Problem identification concerns the lack of immediate, faculty-facing support that connects a text-expressed emotion to a theory-informed response. The objective is to test whether progressively richer routing information improves message quality. Design and development comprise the emotion-classification, SDT-mapping, and message-generation pipeline in Figure 1. Demonstration is provided by the public-corpus benchmark and the synthetic faculty-oriented vignette stress test (Experiments 1–2). Evaluation is provided by the personalization ablation, automatic proxy, blind expert ratings, and multi-level statistical analyses (Experiment 3). Communication is provided by the manuscript and the openly archived code, data-processing scripts, and outputs. This mapping follows the problem–objectives–design/development–demonstration–evaluation–communication sequence described by Peffers et al. [35].

3.2. Datasets

Two public corpora supply the emotion-labeled text used to train and evaluate the classifiers in Experiment 1. GoEmotions [30] contributes 58,000 English Reddit comments, of which single-label rows are retained and regrouped from the original 27 fine-grained categories into the Ekman-6-plus-neutral taxonomy using the mapping released with the dataset. ISEAR [31] contributes roughly 7600 self-reported emotional situations in seven categories; ISEAR is restricted here to the five categories it shares with the Ekman taxonomy (anger, disgust, fear, joy, sadness), dropping shame and guilt, which have no Ekman equivalent. We did not identify a public labeled corpus of naturally occurring higher-education faculty emotional text suitable for this benchmark. Experiment 2 (Section 3.4) therefore uses a researcher-authored synthetic scenario set; it is treated only as a faculty-oriented stress test and not as a sample representative of real faculty communication.

3.3. Emotion Classifiers

Four classifier families of increasing semantic sophistication are evaluated on both corpora:

• nrc_lexicon—word counting against the NRC Emotion Lexicon [36], the same class of method used in the closest prior teacher-emotion work [6]. The predicted label is the emotion with the most lexicon hits in a text; ties default to the majority class.
• tfidf_logreg—a TF-IDF vectorizer (unigrams and bigrams, 20,000 features, sublinear term-frequency scaling) feeding a logistic regression classifier with class-balanced weighting, trained on all seven GoEmotions categories.
• transformer—distilbert-base-uncased [37], fine-tuned for three epochs (batch size 16, learning rate $2\times {10}^{-5}$) on all seven GoEmotions categories.
• llm_fewshot—an API call to Claude Sonnet 4.6 (claude-sonnet-4-6; Anthropic, San Francisco, CA, USA) with five labeled in-context examples per class and no fine-tuning, used purely as a classifier (not as a generator) in this experiment.

Because tfidf_logreg and transformer are trained on all seven GoEmotions categories, they can predict neutral or surprise on the five-category ISEAR and vignette evaluations, where both labels are guaranteed wrong and structurally invisible in a same-label confusion matrix—the corresponding rows simply undercount. We report this explicitly as an out-of-label-space rate (the fraction of predictions falling outside the evaluation label set) and additionally fit tfidf_logreg_5class and transformer_5class, the same algorithms restricted to the five shared categories at training time, as the fair, label-space-matched comparison against nrc_lexicon and llm_fewshot, both of which are already restricted to the evaluation label space by construction. Every out-of-label-space prediction is retained in the denominator and scored as incorrect; no such observation is filtered from the accuracy calculation. Both versions are reported throughout: the seven-category variants as a realistic “what happens if a model trained for a broader context is deployed into a narrower one” baseline, and the five-category variants as the methodologically fair comparison point.

3.4. Faculty Scenario Set

Experiment 2 applies the same six classifier variants to 75 short, first-person vignettes written to operationalize eight stressor categories drawn from the literature on academic-staff burnout and emotion [1,5] and the task structure of the WTMST [24]: workload overload, loss of autonomy, administrative pressure, lack of recognition, isolation, student disengagement, research-teaching conflict, and positive engagement (the last included as a non-distress control). Vignettes were written in sets of 15 per target emotion (anger, disgust, fear, sadness, joy—the five categories shared with ISEAR), each tagged with the emotion it was written to express, a corresponding SDT need, and a stressor category. Table 2 shows how the 75 items distribute across the eight stressor categories; the distribution is intentionally balanced by target emotion (15 per emotion) rather than by stressor category. The set is researcher-authored synthetic test material, not a sample of real faculty communication, and no independent expert content- or face-validity study was completed. Accordingly, Experiment 2 is used only to compare classifier behavior under a faculty-oriented synthetic stress test; no claim of ecological validity, prevalence, or representativeness is made.

Table 2. Stressor-category distribution of the 75 vignettes.

Stressor Category	n (of 75)
Administrative pressure	31
Positive engagement (non-distress control)	15
Isolation	9
Lack of recognition	6
Loss of autonomy	6
Workload overload	4
Research–teaching conflict	3
Student disengagement	1

3.5. SDT Need Mapping and Intervention Generation

The message generator uses the autonomy, competence, and relatedness constructs defined by Self-Determination Theory [10] and applied to teacher motivation in the WTMST [24]. For the artifact, each emotion is mapped heuristically to one of four routing categories: autonomy (anger, disgust), competence (fear, surprise), relatedness (sadness), and reinforcement (joy, treated as a need-satisfied state); neutral input maps to no specific need. Table 3 states the rationale for each design mapping. The specific emotion-to-need pairings are not themselves established SDT measurements or empirically validated diagnostic rules; they are a literature-informed design hypothesis tested only through the downstream ablation and are treated as such in Section 6.

Table 3. Emotion-to-need mapping used by the message generator.

Emotion	SDT Need	Rationale
Anger	Autonomy	Commonly follows a perceived unfair constraint or loss of control over one’s work.
Disgust	Autonomy	Often reflects a values-versus-autonomy conflict with imposed policies or practices.
Fear	Competence	Commonly reflects a perceived threat to one’s adequacy (e.g., evaluation, failure, being replaced).
Surprise	Competence	Often follows unexpected feedback, related to a need for clarity.
Sadness	Relatedness	Commonly linked to social disconnection or lack of recognition.
Joy	Reinforcement	Needs already appear met; the appropriate response is to reinforce, not to address a deficit.
Neutral	None	No clear emotional signal detected.

Four message-personalization levels are generated for every vignette, forming the ablation at the center of Experiment 3:

• generic—an identical message regardless of anything detected, used as the floor condition.
• emotion_only – acknowledges the raw detected emotion in one sentence, with no theoretical framing and no reference to the situation.
• need_only—routes through the mapped SDT need using a small set of hand-written templates phrased in the vocabulary of that need, without referencing the specific situation.
• full_context—generated by an LLM call given the SDT need and the full vignette text, instructed to reference a concrete detail of the situation; there is no template version of this level, since the entire point is content that a small fixed template set cannot express.

Table 4 shows the four levels for two vignettes spanning different emotions and needs, drawn unedited from the experimental output. All four levels were generated in two conditions: oracle, using the gold (researcher-assigned) emotion label, and end_to_end, using the label predicted by llm_fewshot—the best-performing classifier from Experiment 1—together with a need_match_rate diagnostic recording how often a misclassified emotion nonetheless maps to the correct SDT need (anger and disgust, for example, both map to autonomy, so not every classification error degrades the downstream message).

Table 4. Example generated messages by personalization level, for two representative vignettes (drawn unedited from experimental output).

Vignette V1 (anger → autonomy; stressor: workload overload)
“I was assigned three additional committees without being consulted, on top of an already full teaching load.”
generic	Thank you for sharing how things are going. Keep up the great work, you are doing fine!
emotion_only	It’s understandable to feel angry about this.
need_only	It sounds like a decision was made without your input. What is one part of this you could still shape on your own terms?
full_context	Having three committees added to an already full teaching load—without even being asked—is genuinely a lot to carry. Your time and voice matter here. It’s okay to advocate for boundaries; doing so isn’t selfish, it’s sustainable.
Vignette V2 (sadness → relatedness; stressor: isolation)
“Nobody on the faculty has asked how I am doing since I came back from sick leave.”
generic	Thank you for sharing how things are going. Keep up the great work, you are doing fine!
emotion_only	It sounds like that’s been a difficult, disheartening experience.
need_only	Your work seems to be going unnoticed right now, and that is worth naming. Who would you want to know about it?
full_context	Returning from sick leave and feeling invisible to colleagues is genuinely hard—you deserved a warm welcome back. Your presence on that faculty matters, and I hope someone reaches out soon. You shouldn’t have to navigate this return alone.

3.6. Automatic Evaluation

As a computationally inexpensive, reproducible baseline, each generated message is scored by cosine similarity, in a shared TF-IDF space fit jointly over all messages and need descriptions, to the textual description of its target SDT need. Such overlap-based metrics are used here only as an automated proxy and not as a substitute for human evaluation, consistent with known limitations of automatic NLG metrics [22]. Section 4.5 tests directly whether this proxy tracks the blind human ratings.

3.7. Human Evaluation Protocol

The primary outcome of Experiment 3 is a blind expert rating exercise. Thirty higher-education faculty and researchers (Table 5) each rated 25 stratified vignettes (5 per emotion category), with four anonymized message slots per vignette—one per personalization level, randomly ordered and labeled A–D so that raters could not infer which level produced which message—on four dimensions: relevance (does the message respond to what was described), specificity (does it engage with concrete detail rather than generic language), SDT alignment (does it support the need the message was theoretically designed to support, described to raters in plain language rather than as SDT terminology), and motivational usefulness (would the rater expect this message to be motivating if they received it). Each dimension was rated on a prespecified 1–5 scale. During revision, 14 additional post-doctoral researchers completed the same frozen blind instrument; the vignettes, generated messages, randomization scheme, and rating questions were not changed. The final $n=30$ is treated as a feasibility expert sample rather than as a population-representative sample; no population-effect power claim is made. The repeated-measures design instead emphasizes within-rater contrasts, with each rater contributing up to 400 score cells, while inferential tests are conducted at the rater level or with explicit rater and vignette random effects to avoid pseudo-replication. Ratings were collected through Google Forms and exported without names, email addresses, or other direct identifiers.

Table 5. Human-rater composition ($n=30$).

Role	n	Years of Experience
Professor	4	>20
Associate Professor	8	10–20
Assistant Professor	2	5–10
Post-doctoral researcher	14	3–5
Researcher	2	5–10
Total	30	–

Before re-analysis, the raw export was audited for reproducibility. Five records whose complete 400-score profiles duplicated another response were removed, together with one entirely empty record. The export also contained 109 values outside the prespecified 1–5 range (48 zeros and 61 sixes; 0.91% of 12,000 possible score cells); these were treated as missing, with no imputation. The resulting analysis dataset contains 11,891 valid scores from 30 unique raters. The cleaned spreadsheet, audit information, per-rater CSV files, and preprocessing code accompany the repository. This was an expert content-rating exercise on synthetic, machine-generated text, not a study of any individual’s own emotional state, consistent with the determination reported in the Institutional Review Board Statement below.

3.8. Statistical Analysis

The automatic ablation metric (Section 3.6) is item-level by construction (one TF-IDF similarity score per vignette per condition), so its staircase comparisons between adjacent personalization levels (generic vs. emotion_only, emotion_only vs. need_only, need_only vs. full_context) are tested directly with paired t-tests across the 75 vignettes, supported by the Wilcoxon signed-rank test as a non-parametric check, and summarized with Cohen’s d.

The human rubric (Section 3.7) requires a repeated-measures treatment because observations from the same rater are not independent. Three analyses are therefore reported for every adjacent personalization contrast: (i) an item-level paired t-test on available paired rater-by-vignette scores, used only as a descriptive baseline; (ii) a rater-level paired t-test and Wilcoxon signed-rank test, in which each rater’s available item-level scores are averaged within level and dimension and Cohen’s d is computed on the resulting by-rater differences; and (iii) a crossed mixed-effects linear model with personalization level as a fixed effect and crossed random intercepts for rater and vignette, fit by restricted maximum likelihood and releveled to estimate each adjacent contrast directly. The rater-level analysis ($n=30$) remains the primary confirmatory analysis; the mixed-effects model is a model-based robustness check that uses all observed scores. Missing values are not imputed: level means use available ratings, paired tests use available pairs, and mixed models use observed scores.

Inter-rater reliability is quantified formally using two-way random-effects, absolute-agreement intraclass correlations, reported as ICC(2,1) for a single rater and ICC(2,k) for the average of $k=30$ raters, following reporting guidance by Koo and Li [38]. Because ICC requires a complete rater-by-target matrix, it is computed on complete level-by-vignette targets only; the number of retained targets is reported with each estimate. Mean pairwise Pearson correlations are retained as a descriptive supplement. Classifier accuracy is reported against a majority-class baseline throughout, and out-of-label-space predictions remain in the denominator as incorrect predictions.

3.9. Use of Generative AI

Two uses of generative AI are disclosed, and they must not be conflated. The first is an experimental component of the artifact under study, not a manuscript-preparation aid: Claude Sonnet 4.6 (claude-sonnet-4-6; Anthropic) realizes the llm_fewshot classifier and the full_context generator described in Section 3.5; its outputs are data that this paper evaluates, and all prompts and outputs are preserved in the repository for inspection. The second, entirely separate use is a generative AI tool applied only to surface-level language editing of text already drafted by the human authors—correcting grammar, wording, and clarity. This tool was not used to draft, expand, summarize, or restructure any section; to search for, select, or interpret any citation; to propose or evaluate any methodological or statistical choice; or to write, execute, or audit any code. Every substantive decision in this study—study design, the emotion-classifier benchmarking protocol, the literature search and synthesis reported in Section 2, the SDT need-mapping scheme, the human-evaluation rubric and rater recruitment, code implementation and auditing, statistical analysis and interpretation, and source verification—was conceived and carried out solely by the human authors, who take full responsibility for the manuscript and every claim in it. No AI system was treated as an author, consulted for substantive intellectual input, or given any role beyond the narrow language-editing task described here.

4. Results

Results are organized explicitly around RQ1–RQ5. RQ1–RQ2 concern classifier robustness and error patterns; RQ3–RQ5 concern automatic and human evaluation of the four message conditions and the agreement between those evaluation modes. Numerical results are interpreted primarily in terms of what they imply for the design questions rather than repeated from the tables.

4.1. Domain-Shift Ladder

For RQ1, Table 6 and Figure 2 report accuracy for all six classifier variants on three test conditions of increasing distance from the GoEmotions training distribution: in-domain (GoEmotions test split, seven categories), cross-dataset (ISEAR, five shared categories), and a faculty-oriented synthetic stress test (the 75 researcher-authored vignettes, five shared categories). The third condition is not treated as a validated sample of real faculty communication.

Table 6. Classifier accuracy across domain shift. “5class” variants are trained only on the five categories shared with ISEAR and the vignette set.

Model	GoEmotions (In-Domain)	ISEAR (Cross-Dataset)	Faculty Vignettes (Synthetic)
Majority-class baseline	0.350	0.200	0.200
nrc_lexicon	0.384	0.326	0.360
tfidf_logreg (7-class)	0.599	0.259 a	0.227 b
tfidf_logreg_5class (fair)	n/a	0.381	0.387
transformer (7-class)	0.693	0.338 c	0.347 d
transformer_5class (fair)	n/a	0.545	0.520
llm_fewshot	0.467	0.785	0.893

^a 46.5% of predictions fell outside the 5-label evaluation space. ^b 49.3% out-of-label-space. ^c 54.4% out-of-label-space. ^d 52.0% out-of-label-space. All out-of-label-space predictions are retained in the denominator and counted as incorrect; they are not visible in a same-label confusion matrix. Boldface indicates the highest accuracy within each evaluation condition.

Figure 2. Classifier accuracy across three test conditions of increasing distance from the GoEmotions training distribution. “5class” variants are restricted to the five emotion categories shared with ISEAR and the vignette set; the non-suffixed tfidf_logreg and transformer bars are trained on all seven GoEmotions categories and are not label-space-matched to the two out-of-domain evaluations. The faculty-vignette condition is synthetic and independently unvalidated.

Three patterns are most relevant. First, on the label-space-matched comparisons, accuracy rises from lexicon counting to TF-IDF, the fine-tuned transformer, and finally the few-shot LLM on both out-of-domain tests. Second, strong in-domain performance does not guarantee transfer: the seven-class transformer performs well on GoEmotions but sends 54.4% of ISEAR predictions outside the five-label evaluation space. Third, llm_fewshot performs best on both out-of-domain evaluations without fine-tuning, despite being weaker in-domain than the supervised TF-IDF and transformer models. The result supports the methodological value of semantic flexibility under distribution shift, but the 0.893 figure on faculty vignettes should be read only as performance on synthetic researcher-authored material, not as an estimate of accuracy on naturally occurring faculty text.

The label-space correction matters substantively, not just as a footnote. Before restricting tfidf_logreg and transformer to the five shared categories, the original (seven-category) comparison suggested that simple lexicon counting generalized better cross-domain than supervised TF-IDF (0.326 vs. 0.259 on ISEAR). Once both methods are evaluated on a matched label space, the conclusion reverses: tfidf_logreg_5class outperforms nrc_lexicon on both out-of-domain tests (0.381 vs. 0.326 on ISEAR; 0.387 vs. 0.360 on the vignettes). Close to half of the original tfidf_logreg model’s cross-domain predictions (46.5% on ISEAR, 49.3% on the vignettes) fell into label categories—neutral, surprise—that do not exist in either evaluation set and were therefore guaranteed wrong; the earlier, unfair comparison was measuring this artifact at least as much as it was measuring genuine semantic generalization.

4.2. Per-Emotion Error Analysis

For RQ2, Table 7 breaks down accuracy by target emotion on the synthetic vignette set. Disgust is the most consistently difficult category for the non-LLM methods, whereas llm_fewshot improves substantially but still reaches only 60% on this class. Its remaining errors are concentrated in disgust-to-anger confusions, a pattern also seen in weaker methods. Because the vignettes were researcher-authored and independently unvalidated, this pattern should be interpreted as overlap among the constructed scenarios rather than as evidence about naturally occurring faculty language. Figure 3 contrasts the confusion matrices of nrc_lexicon and llm_fewshot.

Table 7. Per-emotion accuracy on the 75-item researcher-authored synthetic vignette set, label-space-fair classifiers only.

Emotion	nrc_lexicon	tfidf_logreg_5class	transformer_5class	llm_fewshot
Anger	0.600	0.467	0.333	0.867
Disgust	0.133	0.000	0.000	0.600
Fear	0.467	0.467	0.733	1.000
Joy	0.467	0.533	0.933	1.000
Sadness	0.133	0.467	0.600	1.000
Overall	0.360	0.387	0.520	0.893

Figure 3. Confusion matrices on the 75-item researcher-authored synthetic vignette set for nrc_lexicon (a) and llm_fewshot (b). Both show disgust-to-anger confusions, while llm_fewshot performs strongly on the other constructed emotion classes.

4.3. Automatic Ablation Results

For RQ3, Table 8 and Figure 4 show that the lexical proxy primarily rewards the emotion_only→need_only step: this contrast is significant in both the oracle and end-to-end conditions, whereas the subsequent need_only→full_context step is not significant and is numerically negative. The proxy therefore favors messages that explicitly share vocabulary with the target need description rather than the richer contextual condition. Exact means and adjacent-step tests are reported in Table 8; Section 4.5 compares this pattern with the blind human evaluation.

Table 8. Automatic TF-IDF similarity-to-need-description proxy by personalization level.

Level	Oracle (Mean Sim.)	End-to-End (Mean Sim.)
generic	0.0229	0.0229
emotion_only	0.0267 (n.s.)	0.0313 (*)
need_only	0.0541 (***)	0.0491 (**)
full_context	0.0455 (n.s.)	0.0461 (n.s.)

Significance markers compare each level to the row immediately above (paired t-test): n.s. $p\ge 0.05$; * $p<0.05$; ** $p<0.01$; *** $p<0.0001$. Exact p-values: generic→emotion_only oracle $p=0.269$, end-to-end $p=0.013$; emotion_only→need_only oracle $p<0.0001$, end-to-end $p=0.005$; need_only→full_context oracle $p=0.083$, end-to-end $p=0.518$.

Figure 4. Automatic TF-IDF-similarity-to-need-description proxy across the four personalization levels, using the gold emotion label ((a), oracle) or the label predicted by llm_fewshot ((b), end-to-end). Error bars show bootstrap confidence intervals. Section 4.5 shows that this metric disagrees with human judgment about the value of the full_context level.

4.4. Human Evaluation Results

For RQ4, after data-quality auditing, the human evaluation contains 30 unique raters and 11,891 valid 1–5 scores (Section 3.7). Figure 5 and Table 9 show that the revised means do not form the previously reported monotonic staircase. The four personalization levels are close on absolute score, with the full-context condition highest for relevance, SDT alignment, and motivational usefulness, while the need-only condition is highest for specificity. The inferential analyses therefore focus on adjacent within-rater contrasts rather than on visual ordering of the raw means.

Figure 5. Mean rater-level expert ratings (1–5 scale; $n=30$) by personalization level and rating dimension. Means are calculated for each rater from available scores and then averaged across raters; no missing values are imputed. The revised data do not show a uniform monotonic increase across personalization levels.

Table 9. Mean of the 30 rater-level means by personalization level and dimension. Missing values are excluded without imputation.

Level	Relevance	Specificity	SDT Alignment	Motivational Usefulness
generic	2.96	2.95	2.96	2.95
emotion_only	2.99	2.95	2.96	3.00
need_only	2.94	3.04	3.00	2.99
full_context	3.10	2.99	3.06	3.10

Boldface indicates the highest mean within each rating dimension.

Neither the generic-to-emotion-only step nor the emotion-only-to-need-only step is significant on any of the four dimensions in the rater-level paired analysis. For the SDT-routing step (emotion_only→need_only), effect sizes are small ($\left|d\right|\le 0.30$) and all paired-t p-values are at least 0.117; the crossed mixed-effects model likewise finds no significant adjacent effect (all $p\ge 0.081$). The revised data therefore do not support a claim that SDT-need-routing alone reliably improves expert-rated message quality over emotion acknowledgement.

The need_only→full_context contrast shows a more selective pattern. At the rater level, relevance increases by 0.155 points ($d=0.37$, paired t $p=0.051$; Wilcoxon $p=0.023$) and motivational usefulness by 0.110 points ($d=0.34$, paired t $p=0.076$; Wilcoxon $p=0.057$). Specificity changes slightly in the opposite direction ($d=-0.09$, $p=0.642$), and SDT alignment shows only a small positive effect ($d=0.15$, $p=0.420$). The crossed mixed-effects model supports the two clearest positive contrasts: full context exceeds need-only for relevance (estimated difference 0.153, $p=0.0038$) and motivational usefulness (0.112, $p=0.030$), but not for specificity ($p=0.398$) or SDT alignment ($p=0.249$). The descriptive item-level analysis gives the same dimensional pattern, with significant gains only for relevance ($p=0.0031$) and motivational usefulness ($p=0.0267$). Thus, full contextualization shows evidence of value, but the effect is dimension-specific and is not uniformly significant across statistical treatments.

Absolute inter-rater agreement is very low. Mean pairwise Pearson correlations across level-by-dimension combinations range from approximately $-0.011$ to 0.021. Formal two-way random-effects absolute-agreement ICCs are likewise near zero: ICC(2,1) ranges from $-0.002$ to 0.006 across the four dimensions, while ICC(2,k) ranges from $-0.077$ to 0.143 (computed on 74–77 complete level-by-vignette targets per dimension). Negative estimates at this magnitude are interpreted as sampling variation around essentially zero agreement, not as meaningful negative reliability. These estimates indicate that absolute 1–5 scores from individual experts are not interchangeable; accordingly, the inferential conclusions rely primarily on within-rater contrasts and crossed models rather than on agreement in absolute score levels.

4.5. Automatic-vs-Human Divergence

For RQ5, the automatic proxy and the human evaluation emphasize different properties of the full_context condition. The automatic metric finds no significant gain over need_only and is numerically lower in both oracle and end-to-end conditions. Human judgments are more nuanced: the full-context condition is not uniformly better, but the revised analyses show its clearest positive signal for relevance and motivational usefulness, including significant mixed-effects contrasts on both dimensions. A plausible mechanical explanation is that the need_only templates share abstract vocabulary with the SDT need descriptions used as the similarity target, whereas full_context messages replace some of that lexical overlap with situation-specific language. This divergence does not establish that all automatic metrics will fail, but it does show that this lexical proxy is not a reliable substitute for human judgment of contextual usefulness.

4.6. Summary of Evidence

Table 10 maps the five research questions stated explicitly in Section 2.5 onto the experiment that addresses each question, the evidence obtained, and the resulting conclusion.

Table 10. Research questions, the experiment addressing each, the evidence obtained, and the resulting conclusion.

Research Question	Experiment	Evidence	Conclusion
RQ1. How does classifier choice affect robustness under domain shift when emotion label spaces are aligned, and to what extent can out-of-label-space predictions distort that comparison?	Exp. 1 (Section 4.1)	Label-space-fair accuracy on GoEmotions/ISEAR/vignettes (Table 6); 46.5–54.4% out-of-label-space rate for unfair 7-class models	On the two out-of-domain evaluations, the fair comparison follows lexicon < TF-IDF < transformer < LLM few-shot; the in-domain ranking differs. The original “lexicon generalizes better” finding was largely a label-space artifact.
RQ2. Which emotion categories remain difficult to classify in the researcher-authored faculty-oriented synthetic vignette set, and how does this vary across classifier families?	Exp. 2 (Section 4.2)	Per-emotion accuracy, four fair classifiers (Table 7)	Disgust is the most consistently difficult category for every non-LLM method; llm_fewshot resolves it best but not perfectly. These results describe the synthetic vignette set only.
RQ3. On a reproducible automatic lexical proxy, does progressively adding emotion information, SDT-need routing, and full situational context improve generated-message quality?	Exp. 3, automatic (Section 4.3)	TF-IDF similarity-to-need-description, oracle & end-to-end (Table 8)	The proxy favors need-only routing and does not reward the full-context condition.
RQ4. In blind expert judgment, does SDT-need routing improve message quality beyond emotion acknowledgement alone, and does full situational context provide a further benefit after rater and item variation are accounted for?	Exp. 3, human (Section 4.4)	30-rater blind rubric; rater-level, crossed mixed-effects, ICC, and descriptive item-level analyses (Figure 6)	SDT routing alone is not consistently supported. Full context shows selective positive evidence for relevance and motivational usefulness, strongest in the mixed-effects analysis, but not a uniform gain across all dimensions or tests.
RQ5. Do the automatic proxy and blind human evaluation agree about the relative value of the four message-personalization conditions?	Exp. 3, both (Section 4.5)	Direct comparison of Table 8 and Table 9	Not fully: the lexical proxy does not reward full context, whereas human ratings show selective gains in relevance and motivational usefulness.

Figure 6. Effect sizes (Cohen’s d for paired rater-level differences, $n=30$) for the three successive personalization steps. Positive values favor the more personalized level. Statistical significance is reported in the text rather than encoded on the effect-size figure. The largest positive effects occur for relevance and motivational usefulness in the SDT-need-to-full-context step.

Table 10. Research questions, the experiment addressing each, the evidence obtained, and the resulting conclusion.

Research Question	Experiment	Evidence	Conclusion
RQ1. How does classifier choice affect robustness under domain shift when emotion label spaces are aligned, and to what extent can out-of-label-space predictions distort that comparison?	Exp. 1 (Section 4.1)	Label-space-fair accuracy on GoEmotions/ISEAR/vignettes (Table 6); 46.5–54.4% out-of-label-space rate for unfair 7-class models	On the two out-of-domain evaluations, the fair comparison follows lexicon < TF-IDF < transformer < LLM few-shot; the in-domain ranking differs. The original “lexicon generalizes better” finding was largely a label-space artifact.
RQ2. Which emotion categories remain difficult to classify in the researcher-authored faculty-oriented synthetic vignette set, and how does this vary across classifier families?	Exp. 2 (Section 4.2)	Per-emotion accuracy, four fair classifiers (Table 7)	Disgust is the most consistently difficult category for every non-LLM method; llm_fewshot resolves it best but not perfectly. These results describe the synthetic vignette set only.
RQ3. On a reproducible automatic lexical proxy, does progressively adding emotion information, SDT-need routing, and full situational context improve generated-message quality?	Exp. 3, automatic (Section 4.3)	TF-IDF similarity-to-need-description, oracle & end-to-end (Table 8)	The proxy favors need-only routing and does not reward the full-context condition.
RQ4. In blind expert judgment, does SDT-need routing improve message quality beyond emotion acknowledgement alone, and does full situational context provide a further benefit after rater and item variation are accounted for?	Exp. 3, human (Section 4.4)	30-rater blind rubric; rater-level, crossed mixed-effects, ICC, and descriptive item-level analyses (Figure 6)	SDT routing alone is not consistently supported. Full context shows selective positive evidence for relevance and motivational usefulness, strongest in the mixed-effects analysis, but not a uniform gain across all dimensions or tests.
RQ5. Do the automatic proxy and blind human evaluation agree about the relative value of the four message-personalization conditions?	Exp. 3, both (Section 4.5)	Direct comparison of Table 8 and Table 9	Not fully: the lexical proxy does not reward full context, whereas human ratings show selective gains in relevance and motivational usefulness.

5. Discussion

The comparative synthesis in Table 10 leads to a narrower theoretical conclusion than the initial submission. The revised human data do not show that SDT-need-routing, by itself, reliably improves message quality over emotion acknowledgement. Instead, the clearest positive evidence appears when the design reaches the full-context condition, and even there it is concentrated in relevance and motivational usefulness rather than all four rating dimensions. This does not invalidate SDT as the conceptual scaffold: autonomy, competence, and relatedness remain well-supported constructs in faculty and teacher motivation research [25,26,27], including recent work on faculty wellbeing in AI-intensive academic settings [13]. The contribution of the SDT layer in this feasibility study is therefore evaluative rather than confirmatory: the ablation makes theory routing explicitly testable and shows that this particular heuristic implementation is insufficient on its own. It does, however, suggest that abstract need labels or templates are insufficient evidence of effective SDT operationalization. The theoretical implication is therefore conditional: need-based theory may be more useful when instantiated in the concrete situation that gives the need meaning, a proposition that requires a stronger factorial design to test causally.

The selective full-context signal is consistent with a separate literature on person-centered supportive communication. A meta-analysis of 23 studies found a positive association between person-centered message features and support outcomes [21], and the “ListenerBot” experiment found benefits from more person-centered computer-delivered support [39]. The present study extends this comparison to faculty-oriented scenarios and LLM-conditioned text, but with an important design caveat: need_only uses hand-written templates whereas full_context uses an LLM. The adjacent contrast therefore cannot isolate situational context from generator mechanism. It should be interpreted as evidence about the full contextualized design condition, not as a pure causal estimate of context alone.

The mechanism generating that situational detail is itself part of a fast-moving line of NLP research on LLM-produced supportive text. Zheng et al. [19] use an LLM to generate richer self-chat training data specifically to make a small, deployable emotional-support chatbot more responsive to situational nuance—a different mechanism aimed at a property similar to the one isolated here. The present results add a controlled, ablation-based estimate of how much that property is actually worth in human judgment relative to theory-routing alone, a question generation-focused work in that literature does not typically ask.

Two methodological results matter beyond this specific pipeline. The first concerns benchmarking practice: the label-space-fairness correction in Experiment 1 reversed an apparent finding (Section 4.1) that a simpler lexicon method generalized better cross-domain than supervised TF-IDF. That apparent finding was, on inspection, almost half artifact—a large fraction of the unfair model’s cross-domain predictions fell into label categories that could not exist in the evaluation set by construction. Anyone building a similar pipeline from public data under resource constraints, where the training label space and the deployment label space rarely match exactly, should expect this kind of artifact by default and report an out-of-label-space rate alongside accuracy, not after the fact.

The second concerns evaluation practice for generated text more broadly. The divergence documented in Section 4.5 between a text-similarity proxy and blind human judgment is a specific instance of a more general, recognized problem: automatic NLG metrics built on lexical or surface overlap are known to correlate poorly with human judgment of quality on exactly the dimensions—specificity, appropriateness to context—that distinguish good generated text from merely fluent generated text [22]. This caution is not specific to lexical-overlap metrics: even an LLM-based evaluator explicitly designed for closer human alignment reaches a Spearman correlation of only about 0.51 with human judgment on a standard summarization benchmark [40], well short of the agreement a deployment decision would ideally rest on. The lesson generalizes past the one metric used here: automatic proxies for generated-text quality, lexical or LLM-based, are useful diagnostic baselines for detecting gross pipeline failures, but not substitutes for the kind of blind rater study this paper treats as primary.

For institutional practice, this study speaks most directly to faculty development and human-resources functions, not to clinical or counseling services. Prior work links faculty stress and emotion-management strategies to wellbeing and productivity [5], while intervention reviews emphasize the heterogeneity of faculty-support approaches [2,3]. Recent faculty–AI studies further connect technology adoption or AI literacy with stress, wellbeing, and satisfaction [12,13]. Against that literature, the present study contributes a narrower design implication: inserting a theoretical label into an automated response pipeline is not enough to demonstrate usefulness. A faculty-support tool should be evaluated at the level of the actual message received, should privilege contextual relevance over nominal “personalization,” and should remain one input among workload management, peer support, and existing wellbeing services [1].

A specific risk deserves explicit treatment, because it is the most likely way a tool like this would be misused: routing a detected emotion to an automatically generated supportive message is not, and should not be marketed or deployed as, automated psychological diagnosis or therapy. A recent evaluation of LLM-based systems explicitly positioned as therapists found that they can express stigma toward mental health conditions and respond inappropriately or unsafely in crisis scenarios, with no clear improvement from newer or larger models [41]. A future implementation should therefore place a separate, high-sensitivity crisis gate before the emotion-to-SDT pipeline. Inputs would first be screened by an independently validated crisis detector combining explicit high-risk lexical rules with a calibrated semantic classifier. If either component flags explicit self-harm intent, imminent danger, or severe psychological distress above a deployment-calibrated threshold ${\tau }_{\mathrm{crisis}}$, the automated motivational generator must be bypassed and the case routed immediately to a designated human support pathway. Ambiguous cases should also be held for human review. Only inputs below this gate should proceed to ordinary emotion classification and message generation; a routine complaint about administrative pressure would therefore remain in the normal path unless it also contains a crisis signal. The threshold should be selected for high sensitivity and validated prospectively before deployment, not inferred from the present vignette data. The tool’s stated function should remain narrow, its output should not be presented as a clinical judgment, and individual-level emotional profiles should not be retained beyond explicit consent.

That practical reading carries the same caveat already noted in the Results: the 0.893 classifier accuracy was obtained on the researcher-authored synthetic vignette set and cannot be interpreted as real-world faculty accuracy. The personalization pipeline has likewise been tested on synthetic scenarios rated by external experts, not in live interaction with faculty describing their own experiences. This study therefore provides feasibility evidence about design alternatives, not evidence that the system is ready to deploy.

6. Limitations and Future Work

Several limitations bound how far these results should be generalized.

No naturally occurring faculty emotional text was collected or used in Experiments 2–3. The 75-item scenario set is researcher-authored and was not independently content- or face-validated. It should therefore be treated strictly as synthetic stress-test material for comparing methods, not as evidence about the prevalence, distribution, wording, or ecological validity of real faculty emotional expression. Independent expert validation of the existing scenarios, followed by evaluation on ethics-approved naturally occurring faculty data, is required before any ecological-validity claim can be made.

The emotion-to-SDT-need mapping (Table 3) is a literature-informed design proposal, not a validated psychological instrument. The revised human results do not provide independent validation of that mapping: the need-only condition does not reliably improve ratings over emotion-only messages. Future work should therefore separate validation of the mapping itself (e.g., organizational-psychology expert review and faculty self-report) from evaluation of the generated message.

Inter-rater agreement on absolute scores is very low. Mean pairwise correlations are approximately zero, ICC(2,1) ranges from $-0.002$ to 0.006, and ICC(2,k) from $-0.077$ to 0.143 across dimensions. The rater pool is also a convenience sample rather than an internationally representative sample. These findings limit any interpretation of absolute mean scores and reinforce the use of within-rater contrasts and crossed models; they do not remove the need for a larger, more diverse sample. Future rating protocols should provide behaviorally anchored descriptions for each scale point, worked calibration examples, and a brief training/pilot phase before formal scoring [38].

The divergence between automatic and human evaluation (Section 4.5) was diagnosed for one specific similarity-based metric computed against one specific kind of reference text (a short need description). The argument that automatic metrics are an unsafe substitute for human judgment in this domain almost certainly generalizes beyond this exact metric, but alternative automatic metrics (e.g., neural embedding similarity) that might partially close the gap were not tested, and this metric is not claimed to be uniquely bad among automatic options.

Generative-model limitations also remain. The llm_fewshot classifier and full_context generator can inherit model-specific biases, produce unsupported assumptions about a person’s situation, or generate fluent but inappropriate content. The current experiment evaluates message quality but is not a comprehensive hallucination, fairness, or safety audit. In addition, the need_only and full_context conditions differ both in contextual information and in generation mechanism (template versus LLM), so their contrast does not identify a pure causal effect of context. Future factorial experiments should cross theory-routing, access to situation text, and generation mechanism independently, and should include dedicated bias and safety stress tests.

Finally, this is a feasibility study about personalization design, not validation of a deployable system. A staged follow-up program should therefore (1) independently validate the existing vignette set, (2) validate or revise the emotion-to-need mapping with domain experts and faculty self-report, (3) repeat the blind rating study with behaviorally anchored scales and a more diverse rater pool, (4) separate context and generation mechanism in a factorial ablation, (5) conduct dedicated bias, hallucination, and crisis-safety testing, and only then (6) proceed to an ethics-approved live faculty pilot.

7. Conclusions

This study evaluated a theory-routed digital faculty-support design through a domain-shift-aware classifier benchmark, a researcher-authored synthetic vignette stress test, and blind ratings from 30 higher-education faculty and researchers. The revised analysis does not support a consistent human-rated benefit from SDT-need-routing alone. The full-context condition shows a more promising but selective signal: relative to need-only messages, relevance and motivational usefulness improve in the crossed mixed-effects analysis, whereas specificity and SDT alignment do not, and the rater-level paired tests are not uniformly significant. The lexical automatic proxy does not capture these selective human-rated gains. These findings support continued study of contextualized, theory-informed message design, but they do not establish ecological validity or deployment effectiveness because the faculty vignettes were synthetic and independently unvalidated, the emotion-to-need mapping is a design heuristic, and absolute inter-rater agreement was very low. The next substantive step is an ethics-approved empirical program that validates the stimulus material and mapping, improves rater calibration, separates contextualization from generator effects, audits bias and crisis safety, and ultimately tests the system with real faculty using validated instruments such as WLEIS [8], TEIQue-SF [9], and WTMST [24].