CPU-Only Self Enhancing Authoring Copilot Design-Based Markov Decision Processes Orchestration and Qwen 3 Local Large Language Model

Abstract

We introduce a novel, privacy-preserving AI authoring copilot designed for educational content creation, which uniquely combines a Markov Decision Process (MDP) as a reinforcement learning orchestrator with a locally deployed Qwen3-1.7B-ONNX large language model to iteratively refine text for clarity, unity, and engagement—all running on a modest CPU-only system (Intel i7, 16 GB RAM). Unlike cloud-dependent models, our agent treats writing as a sequential decision problem, selecting refinement actions (e.g., simplification, elaboration) based on real-time LLM and sentiment feedback, ensuring pedagogically sound outputs without internet dependency. Evaluated across five diverse topics, our MDP-orchestrated agent achieved an overall average quality score of 4.23 (on a 0–5 scale), statistically equivalent to leading cloud-based LLMs like ChatGPT and DeepSeek. This performance was validated through blind evaluations by four independent LLMs and human raters, supported by statistical consistency analysis. Our work demonstrates that lightweight local LLMs, when guided by principled MDP policies, can deliver high-quality, context-aware educational content, bridging the gap between powerful AI generation and ethical, on-device deployment. This advancement empowers educators, researchers, and curriculum designers with a trustworthy, accessible tool for intelligent content augmentation aligning with the Quality Education Sustainable Development Goal through innovations in educational technology, inclusive education, equity in education, and lifelong learning.

1. Introduction

The rapid evolution of artificial intelligence has moved beyond mere automation, entering a new era of intelligent augmentation, where AI systems do not simply generate text but actively refine, adapt, and enhance it with purpose. In educational contexts—where clarity, engagement, and accessibility are critical to effective knowledge transfer—the demand for such intelligent support has never been greater. Teachers, curriculum designers, and academic researchers frequently face the challenge of distilling complex ideas into coherent, pedagogically sound content, often under tight time constraints. While large language models (LLMs) such as ChatGPT and DeepSeek have demonstrated remarkable generative capabilities, their outputs often suffer from inconsistencies in tone, structure, and rhetorical coherence, necessitating extensive manual editing. More importantly, their reliance on cloud-based infrastructure raises significant concerns regarding data privacy, latency, and lack of control—barriers that hinder their integration into sensitive or real-time educational settings.

To address these limitations, this paper introduces a novel AI authoring agent designed specifically for educational content creation. The system integrates advanced AI methodologies into a unified, on-device framework: Markov Decision Processes (MDPs) for orchestrating iterative refinement actions, a locally deployed Qwen3-1.7B model in ONNX format for efficient and private inference, and sentiment analysis for real-time quality assessment. Unlike conventional “prompt-and-generate” approaches, our agent treats content improvement as a sequential decision-making process. Each action—such as enhancing clarity, improving unity, or adjusting tone—is selected by an MDP policy that maximizes long-term quality metrics, ensuring coherent and purposeful evolution of the text. This decision-theoretic foundation enables the agent to reason over trade-offs, maintain narrative consistency, and adapt dynamically to user-defined goals.

The agent is designed to empower diverse educational stakeholders: teachers can rapidly generate lesson summaries and personalized feedback, researchers can refine technical writing for broader audiences, and curriculum developers can produce adaptive, emotionally resonant materials. By operating entirely on-device through ONNX optimization, the system guarantees data privacy and low-latency interaction—key requirements for deployment in classrooms and field settings. We evaluate its performance across multiple topics, comparing outputs from our agent with those from leading LLMs. To ensure objective assessment, we employ multiple LLMs—ChatGPT, Copilot, Qwen, and DeepSeek—as blind evaluators, rating generated paragraphs on clarity, unity, and overall quality. Results show that the MDP-orchestrated agent consistently produces content that is more coherent, pedagogically effective, and engaging, underscoring the advantages of combining structured reasoning with generative power.

Despite growing interest in combining reinforcement learning with large language models for structured generation, existing approaches primarily focus on code synthesis or task completion in cloud-based settings, with limited attention to pedagogical text refinement or on-device privacy-preserving operation. Crucially, these works do not explicitly map MDP components (states, actions, rewards) to measurable linguistic attributes like clarity, unity, or sentiment—nor do they enforce iterative quality control grounded in educational best practices. In contrast, our work introduces a quantitatively defined orchestration mechanism where (1) states correspond to discrete stages of textual evaluation (e.g., initial production, clarity check, unity enhancement); (2) actions are linguistically grounded operations (e.g., simplification, elaboration, coherence repair); and (3) rewards are computed from interpretable, sentiment-augmented LLM assessments of specific rhetorical dimensions. We hypothesize that this tight coupling between decision-theoretic control and localized linguistic feedback enables more coherent, engaging, and pedagogically sound outputs than single-pass or cloud-dependent alternatives.

This work contributes to the emerging paradigm of human-centered AI in education—not as a replacement for human expertise, but as a cognitively aware co-author that enhances the capabilities of educators and knowledge creators through technically robust, ethically responsible, and pedagogically intelligent support. The paper is organized as follows: Section 2 reports the related works while Section 3 presents the materials and methods, detailing the architecture of the agent, the use of LLMs, and the MDP-based orchestration mechanism. Section 4 reports the experimental results and evaluation outcomes. Finally, Section 5 concludes the paper by summarizing key findings and discussing future directions.

2. Related Work

Recent advances in AI-driven content generation have demonstrated the growing role of large language models (LLMs) and conversational agents in educational and creative domains. Yusuf et al. [1] introduced a framework for pedagogical AI agents in higher education, highlighting their potential in personalized instruction and administrative automation, while Schorcht et al. [2] showed how communicative agents can collaboratively refine mathematical tasks through iterative dialogue. These works emphasize the adaptability of AI agents in structured workflows, yet they primarily focus on task-specific applications rather than long-form, coherent content creation. Akçapınar and Sidan [3] further explored the dual role of AI chatbots in programming education, finding that while LLMs enhance productivity, they may encourage over-reliance without sufficient safeguards—raising concerns about quality control and pedagogical integrity. This reveals a critical gap: current systems lack mechanisms to ensure alignment with domain-specific best practices during generative processes, especially in complex, multi-stage tasks like eBook authoring.

Markov Decision Processes (MDPs) offer a principled approach to sequential decision-making in dynamic environments and have been successfully applied in planning and control. Sarsur et al. [4] used MDPs with modular supervisors for efficient planning, while Chen et al. [5] demonstrated their effectiveness in robotic path planning under uncertainty. Bäuerle and Jaśkiewicz [6] extended MDPs to risk-sensitive settings, enabling robust policy optimization. However, these applications are largely confined to physical or abstract decision spaces and do not incorporate semantic knowledge. Our work bridges this gap by embedding MDPs within a knowledge map, where states represent stages of the writing process and actions correspond to LLM-driven refinements guided by best practices.

Knowledge graphs (KGs) provide a structured means of encoding domain expertise, enhancing both interpretability and reasoning in AI systems. Ibrahim et al. [7] surveyed KG–LLM integration, demonstrating improved transparency in decision-making, while Schenker et al. [8] pioneered graph-based web mining by leveraging topological features for classification. More recently, Khan et al. [9] proposed a Distance-Driven GNN to mitigate feature sparsity using global structural information. Although these works highlight the potential of KGs, they primarily treat them as static knowledge repositories rather than dynamic process guides.

The integration of large language models (LLMs) into structured workflows has gained significant traction, especially in domains like code generation and human–agent collaboration. Fernandes et al. [10] demonstrated DeepSeek-R1’s strong reasoning capabilities in generating correct LoRaWAN code, while Ghamati et al. [11] introduced a personalized LLM for context-aware human–robot interaction. However, these approaches typically depend on cloud-based models, which can introduce latency, raise privacy concerns, and offer limited fine-grained control. In contrast, our framework operates with a locally hosted instance of DeepSeek-R1, fine-tuned to adhere to the constraints derived from the knowledge graph discussed in the literature review (though the graph itself is not used in our implementation). To address well-documented fine-tuning challenges—such as catastrophic forgetting and reward misalignment, as highlighted by Wu et al. [12]—we employ MDP-driven reinforcement learning that explicitly prioritizes outputs conforming to the graph-defined quality standards.

Designing trustworthy and usable agentic AI remains a challenge. Diebel et al. [13] found that excessive autonomy can reduce users’ perceived competence, underscoring the need for balanced human–AI collaboration. Catak and Kuzlu [14] quantified LLM uncertainty via convex hull analysis, revealing sensitivity to prompt complexity—a vulnerability our iterative MDP-based validation loop helps mitigate. Aylak [15] proposed SustAI-SCM for sustainable supply chains, but its static workflows lack the adaptability required for creative tasks. Zota et al. [16] advanced hyper-automated IT operations, though their focus on incident resolution does not extend to the nuanced demands of content generation.

Our work introduces a novel CPU-only, privacy-preserving AI authoring copilot that uniquely combines Markov Decision Process (MDP)-driven orchestration with a locally deployed Qwen3-1.7B-ONNX large language model. Unlike prior research—which either lacks formal sequential decision-making frameworks for text refinement, relies on cloud-based models, or targets non-pedagogical domains—our system operates entirely on modest consumer hardware (Intel i7 CPU, 16 GB RAM) without GPU acceleration or internet dependency. This enables secure, on-device generation and iterative enhancement of educational content, grounded in measurable quality metrics such as clarity, unity, and sentiment. The

Table 1. Comparison of our approach with related works.

Work	Educational Context	Use of MDP	Local LLM	Padagogical Text
Yusuf et al. [1]	Yes	No	No	Yes
Schorcht et al. [2]	Yes	No	No	No
Sarsur et al. [4]	No	Yes (non-text)	No	No
Chen et al. [5]	No	Yes (robotics)	No	No
Fernandes et al. [10]	No	No	Yes	No
Our Work	Yes	Yes	Yes	Yes

below systematically contrasts our contribution against representative works in the literature across three key criteria: (1) use of an MDP to guide text refinement, (2) deployment of a local/on-device LLM, and (3) successful execution on CPU-only hardware with less than 16 GB RAM.

3. Materials and Methods

This study presents a structured AI agent design that integrates pre-trained AI models as skill providers, and Markov Decision Process (MDP) for orchestration. The approach aims to create an intelligent system capable of reasoning, executing tasks, and optimizing its actions through sequential decision-making. This section is organized into distinct areas covering the role of graph-based knowledge representation in structuring AI understanding, the integration of pre-trained models as skill providers for language generation and evaluation, and the use of MDP to orchestrate tasks efficiently.

3.1. Materials

We used a laptop with an Intel i7 CPU, 16 GB RAM, and 1 TB SSD to host the Qwen3-1.7B-ONNX model for processing and the bert-base-multilingual-uncased-sentiment model for sentiment analysis, adapted via JavaScript to output an integer from zero to five stars. Both models were converted to ONNX for web integration. This setup provided sufficient power and storage for efficient and stable system performance.

3.2. Methods

This section describes the orchestration engine that controls our authoring copilot. The Markov Decision Process (MDP) framework is employed to guide the sequential generation of paragraphs, ensuring coherence, engagement, and uniform reading flow across the article. Within this approach, states represent different stages in structuring content, actions correspond to stylistic or structural choices, and rewards reflect improvements in clarity and logical progression. By systematically transitioning between states, the MDP optimizes the ordering and composition of paragraphs, ensuring that each section builds upon the previous one in an intuitive and compelling manner. The framework enhances readability by dynamically adjusting the writing flow, reinforcing thematic consistency, and maintaining a balance between depth and accessibility. We model the chapter writing process as a finite MDP with quality-weighted rewards. The model captures the trade-offs between different writing quality dimensions during paragraph composition.

3.2.1. State Space

Let’s define $\mathcal{S}=\{S_P,S_C,S_{EC},S_{EU},S_D\}$ as the set of MDP states. $S_P$ represents production state that triggers the MDP to produce a paragraph p or deliver the whole chapter and close the process. $S_C$ represents the check state on which the MDP checks the clarity and unity of the produced paragraph p. $S_{EC}$ is the state that marks that the paragraph p clarity was enhanced while $S_{EU}$ state marks that the paragraph p unity was enhanced. $S_D$ marks the end of the process and flags the paragraph delivery.

3.2.2. Action Space

This subsection defines the action space, where structural choices guide transitions between states defined above as the following:

• $P\left(p\right)$: Produce the paragraph p (From $S_C$ state).
• $C\left(p\right)$: Check the paragraph p (From $S_P$ state).
• $E_c\left(p\right)$: Enhance the clarity of the paragraph p (From $S_C$ state).
• $E_u\left(p\right)$: Enhance the unity of the paragraph p (From $S_C$ state).
• $D\left(p\right)$: Deliver the paragraph p (From $S_C$ state).

3.2.3. MDP State Machine

This subsection focuses on the Markov Decision Process (MDP) state machine that governs the behavior of our authoring agent. The MDP is defined over a finite set of states $\mathcal{S}=\{S_P,S_C,S_{EC},S_{EU},S_D\}$, representing key stages in the paragraph generation and refinement process. Transitions between these states are driven by a structured action space, including paragraph production $P\left(p\right)$, clarity and unity checks $C\left(p\right)$, and enhancement actions $E_c\left(p\right)$ and $E_u\left(p\right)$. The state machine orchestrates the flow from initial production $S_P$ through iterative evaluation and refinement in $S_C$, with optional detours to $S_{EC}$ and $S_{EU}$ for targeted improvements, until the final delivery state $S_D$ is reached, concluding the authoring process. The diagram visually captures these transitions, illustrating how the agent navigates through states to produce a polished, coherent paragraph.

 

3.2.4. Transition Probabilities

This subsection defines the transition functions taking into consideration the states and action spaces defined above. Transition function defines how the agent will move from a state to another.

• $T(S_P\mid S_C,P\left(p\right))={\delta }_{p=\mathrm{null}}$: This transition represents the probability of moving from the check state $S_C$ back to the production state $S_P$ by performing the action $P\left(p\right)$—producing a new paragraph p. The Kronecker delta ${\delta }_{p=\mathrm{null}}$ ensures that this transition occurs only if no valid paragraph has been produced (i.e., p is null). Thus, the agent returns to production when the current paragraph is incomplete, undefined or in the first time when we still do not have any paragraph yet ($p=null$).
• $T\left(S_C\right|S_P,C\left(p\right))=1$: After production, we always move to checking state. During the check state, a check action $C\left(p\right)$ is performed to evaluate the clarity and unity of the paragraph p, and then, the reward is computed.
• $T\left(S_{EC}\right|S_C,E_c\left(p\right))\sim \mathrm{Bernoulli}\left({\alpha }_c\right)$: Represents the probability to move from $S_{EC}$ when we are in $S_C$ by performing the action $E_c\left(p\right)$, enhancing the clarity of the paragraph p when the clarity check is less than the ceil $c_0$ (it is 4 out of 5 in our case).
• $T\left(S_{EU}\right|S_C,E_u\left(p\right))\sim \mathrm{Bernoulli}\left({\alpha }_u\right)$: Represents the probability to move from $S_{EU}$ when we are in $S_C$ by performing the action $E_u\left(p\right)$, enhancing the unity of the paragraph p when the clarity check is less than the ceil $u_0$ (it is 4 out of 5 in our case).
• After enhancing the clarity or the unity, the agent comes back to check the paragraph p by acting $C\left(p\right)$ and coming back to $S_C$ from $S_{EC}$ and $S_{EU}$, respectively. Since the transition is automatic and deterministic, then we conclude that $T\left(S_C\right|S_{EC},C\left(p\right))=1$ and $T\left(S_C\right|S_{EU},C\left(p\right))=1$.
• $T(S_D\mid S_C,D\left(p\right))\sim \mathrm{Bernoulli}\left(0.33\right)$: This transition represents the probability of moving from the check state $S_C$ to the delivery state $S_D$ by performing the action $D\left(p\right)$—delivering the paragraph p. Delivery occurs when the evaluations of clarity and unity during $S_C$ exceed predefined thresholds $c_0$ and $u_0$, respectively. At $S_C$, the agent has three equally likely outcomes: (1) deliver the paragraph, (2) enhance clarity, or (3) enhance unity. Assuming uniform likelihood among these decisions, each has a probability of approximately $1/3\approx 33\%$, justifying the use of a Bernoulli distribution with mean $0.33$ for this discrete choice in the transition model. Although we might achieve the clarity threshold in iteration n, we may then focus on improving unity in iteration $n+1$. However, this adjustment can unintentionally disrupt the previously satisfied clarity criterion. Consequently, even if clarity was met in a prior step, it must be re-verified in every subsequent iteration. This continual need for re-evaluation introduces a persistent, uniform uncertainty across all iterations, as no outcome remains guaranteed once other dimensions are modified.

3.2.5. Reward Function

The reward function is designed to guide the authoring agent toward producing high-quality paragraphs by providing feedback based on key textual attributes: clarity, unity, and sentiment. Specifically, the reward is defined as a weighted sum of quality metrics evaluated through a combination of LLM-based assessment and sentiment analysis. Let ${\mathcal{L}}_{\mathrm{clarity}}\left(p\right)$ and ${\mathcal{L}}_{\mathrm{unity}}\left(p\right)$ denote the evaluations of paragraph p’s clarity and unity, respectively, performed by the LLM Qwen. These evaluations are further processed by a sentiment analysis model $\psi$ (bert-base-multilingual-uncased-sentiment in our case), which maps textual quality to a numerical score ranging from 0 (low sentiment/quality) to 5 (highly positive sentiment/quality). The composed function $\psi \circ \mathcal{L}\left(p\right)$ thus quantifies the perceived quality of the paragraph: that means that the sentiment analyser will analyze the answer of the LLM about the clarity of the paragraph p. This answer is given by $\mathcal{L}\left(p\right)$ (the same for the unity). The reward then is defined as

$$R(s,a)=\left\{\begin{array}{cc}{w}_c·\psi \circ {\mathcal{L}}_{\mathrm{clarity}}\left(p\right)+w_u·\psi \circ {\mathcal{L}}_{\mathrm{unity}}\left(p\right)\hfill & \mathrm{during}\mathrm{the}\mathrm{check}\mathrm{action}\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(1)

where $w_c$ and $w_u$ are configurable weights that balance the importance of clarity and unity during enhancement actions. This formulation ensures that only enhancement actions—those that improve the text—receive non-zero rewards, encouraging the agent to refine the paragraph until it meets desired quality standards. In our experiments, we explicitly set the default weight values to 0.6 for clarity and 0.4 for unity to reflect their relative importance in the evaluation framework. Users are free to adjust these values according to their specific priorities or contextual requirements.

3.3. Q-Function Definition

The Q-function, $Q(s,a)$, represents the expected cumulative reward for taking action a in state s and following an optimal policy thereafter. It is defined by the Bellman Equation (2).

$$Q(s,a)=R(s,a)+\gamma \underset{a^{\prime }}{max}\sum _{s^{\prime }}T\left(s^{\prime }\right|s,a)Q(s^{\prime },a^{\prime })$$

(2)

where $R(s,a)$ is the immediate reward for action a in state s, $\gamma$ is the discount factor, $T\left(s^{\prime }\right|s,a)$ is the transition probability, and ${max}_{a^{\prime }}Q(s^{\prime },a^{\prime })$ estimates the best future reward. The reward function guides the agent toward high-quality text. Q-learning (Algorithm 1) updates Q-values using this rule, with an $ϵ$-greedy policy balancing exploration and exploitation.

Algorithm 1: Q-Learning algorithm with $ϵ$-greedy policy

3.4. Q-Learning Algorithm

This Algorithm 1 allows reinforcement learning agents to refine their decision-making process while balancing clarity, engagement, and flow coherence. In Algorithm 1, we fix the learning rate $\alpha =0.8$, discount factor $\gamma =0.7$, and exploration rate $\epsilon =0.8$ based on empirical pilot experiments. These values reflect a deliberate trade-off between rapid policy convergence and sufficient exploration of the refinement action space.

A high learning rate ($\alpha =0.8$) enables the Q-function to quickly incorporate feedback from LLM-based quality assessments—an important feature given the strict latency constraints of on-device operation and the limited number of refinement iterations per paragraph (typically $\le 3$).

Although aggressive learning rates can destabilize training under noisy rewards, our reward function combines deterministic LLM evaluations with sentiment analysis which stabilizes learning despite the high $\alpha$. The moderate discount factor ($\gamma =0.7$) prioritizes immediate improvements in clarity and unity while still accounting for cumulative effects across sequential edits.

3.5. Implementation

We designed an authoring agent governed by a Markov Decision Process (MDP) to generate a paragraph and iteratively enhance its clarity and unity. The agent first produces an initial draft and then enters a refinement loop, where the paragraph is evaluated and improved based on learned quality metrics. The agent is developed using Node.js platform and leverages the transformers.js to load the Qwen3-1.7B model locally. When deploying large language models like Qwen3-1.7B in a server-side JavaScript environment, leveraging the ONNX format can significantly enhance inference performance and cross-platform compatibility. ONNX (Open Neural Network Exchange) enables models trained in frameworks like PyTorch or TensorFlow to be exported into a standardized, optimized representation that can be executed efficiently using high-performance runtimes such as ONNX Runtime. For server-side applications built with Node.js, this is particularly valuable because it allows access to hardware-accelerated inference (via CPU optimizations or GPU backends like CUDA) without relying on Python-based serving stacks. While Transformers.js is primarily designed for browser and lightweight Node.js use cases, it does not natively support ONNX execution. Instead, Transformers.js uses its own WebAssembly (WASM) or native JavaScript tensor operations, which are convenient but generally slower than optimized ONNX runtimes for larger models like Qwen3-1.7B. Therefore, to achieve optimal performance when serving Qwen3-1.7B on the server, it is recommended to bypass Transformers.js for inference and instead integrate ONNX Runtime for Node.js (onnxruntime-node) directly. This package provides bindings to the native ONNX Runtime, enabling fast CPU inference (with optional AVX2/AVX-512 acceleration) and support for quantized models. To evaluate the effectiveness of our approach, we selected five diverse topics (see prompts in Appendix A). For each topic, we generated a paragraph using our MDP-based agent, DeepSeek, ChatGPT, Copilot X, Grok-3 and our MDP Agent using identical prompts to ensure a fair comparison. This resulted in a total of 25 paragraphs—five from each system. All paragraphs were anonymized and assigned unique identifiers to prevent bias during evaluation. We then employed several state-of-the-art large language models such as OpenAI’s ChatGPT, Alibaba’s Qwen, and DeepSeek as independent evaluators, in addition to human evaluators. Each LLM was instructed to rate the clarity and unity/completeness of every paragraph on a 5-point scale, from 0 (very poor) to 5 (excellent), without knowledge of the generating model.

4. Results and Discussion

4.1. Obtained Paragraphs

For completeness and transparency, all generated paragraphs—covering the five topics—are included in Appendix B. Each paragraph is presented with its corresponding identifier, topic, and generating model (ChatGPT, DeepSeek, or the MDP-based agent), allowing for direct inspection and qualitative assessment of the content. This enables readers to contextualize the evaluation scores and to examine the stylistic and structural differences among the systems. Now, we will ask existing LLMs to evaluate the clarity and unity of all generated paragraphs. See the evaluation results in the section below.

4.2. Evaluation Results

Evaluation results from ChatGPT (GPT-4o), Qwen (Qwen3), and DeepSeek (DeepSeek-V3) are presented in

Table 2. Clarity and unity ChatGPT evaluation results of generated paragraphs.

Paragraph ID	Clarity Score	Unity Score	Average Score
Paragraph 11	4.3	4.4	4.35
Paragraph 12	4.2	4.4	4.30
Paragraph 13	4.3	4.5	4.40
Paragraph 14	4.4	4.2	4.30
Paragraph 15	4.3	4.3	4.30
Paragraph 21	4.4	4.5	4.45
Paragraph 22	4.3	4.2	4.25
Paragraph 23	4.4	4.4	4.40
Paragraph 24	4.3	4.3	4.30
Paragraph 25	4.4	4.3	4.35
Paragraph 31	4.2	4.1	4.15
Paragraph 32	4.1	4.0	4.05
Paragraph 33	4.2	4.4	4.30
Paragraph 34	4.3	4.1	4.20
Paragraph 35	4.3	4.2	4.25
Paragraph 41	4.4	4.4	4.40
Paragraph 42	4.3	4.3	4.30
Paragraph 43	4.4	4.5	4.45
Paragraph 44	4.3	4.3	4.30
Paragraph 45	4.4	4.4	4.40
Paragraph 51	3.8	3.9	3.85
Paragraph 52	4.0	4.1	4.05
Paragraph 53	4.2	4.4	4.30
Paragraph 54	3.9	4.0	3.95
Paragraph 55	4.3	4.3	4.30
Average	4.25	4.29	4.27

,

Table 3. Clarity and unity Qwen evaluation results of generated paragraphs.

Paragraph ID	Clarity Score	Unity Score	Average Score
Paragraph 11	4.4	4.3	4.35
Paragraph 12	4.2	4.1	4.15
Paragraph 13	4.3	4.2	4.25
Paragraph 14	4.5	4.0	4.25
Paragraph 15	4.3	4.2	4.25
Paragraph 21	4.3	4.2	4.25
Paragraph 22	4.1	4.0	4.05
Paragraph 23	4.0	4.1	4.05
Paragraph 24	4.2	3.9	4.05
Paragraph 25	4.2	4.1	4.15
Paragraph 31	4.0	4.0	4.00
Paragraph 32	3.9	3.8	3.85
Paragraph 33	4.1	4.2	4.15
Paragraph 34	4.3	4.0	4.15
Paragraph 35	4.0	4.1	4.05
Paragraph 41	4.4	4.3	4.35
Paragraph 42	4.2	4.1	4.15
Paragraph 43	4.2	4.3	4.25
Paragraph 44	4.4	4.0	4.20
Paragraph 45	4.3	4.2	4.25
Paragraph 51	3.8	4.2	4.00
Paragraph 52	4.1	4.0	4.05
Paragraph 53	4.1	4.2	4.15
Paragraph 54	4.3	3.8	4.05
Paragraph 55	4.2	4.2	4.20

and

Table 4. Clarity and unity DeepSeek evaluation results of generated paragraphs.

Paragraph ID	Clarity Score	Unity Score	Average Score
Paragraph 11	4.2	4.3	4.25
Paragraph 12	4.4	4.4	4.4
Paragraph 13	4.3	4.3	4.3
Paragraph 14	4.2	4.2	4.2
Paragraph 15	4.3	4.3	4.3
Paragraph 21	4.2	4.3	4.25
Paragraph 22	4.3	4.3	4.3
Paragraph 23	4.2	4.2	4.2
Paragraph 24	4.1	4.2	4.15
Paragraph 25	4.2	4.2	4.2
Paragraph 31	4.1	4.0	4.05
Paragraph 32	4.2	4.2	4.2
Paragraph 33	4.2	4.3	4.25
Paragraph 34	4.2	4.3	4.25
Paragraph 35	4.1	4.1	4.1
Paragraph 41	4.3	4.3	4.3
Paragraph 42	4.4	4.4	4.4
Paragraph 43	4.3	4.4	4.35
Paragraph 44	4.2	4.2	4.2
Paragraph 45	4.3	4.3	4.3
Paragraph 51	4.0	4.5	4.25
Paragraph 52	4.3	4.4	4.35
Paragraph 53	4.2	4.3	4.25
Paragraph 54	4.1	4.0	4.05
Paragraph 55	4.2	4.3	4.25

, respectively. Each table compares the performance of paragraphs generated by five models, including our MDP generator (running on CPU with low memory) in ChatGPT, DeepSeek, Copilot and Grok.

4.2.1. Consolidated Evaluation Summary Across Models

To facilitate cross-model comparison, the following

Table 5. Cross-model average score comparison overview.

Paragraph ID	ChatGPT	DeepSeek	Qwen	Overall Average
11	4.35	4.25	4.35	4.32
12	4.30	4.40	4.15	4.28
13	4.40	4.30	4.25	4.32
14	4.30	4.20	4.25	4.25
15	4.30	4.30	4.25	4.28
21	4.45	4.25	4.25	4.32
22	4.25	4.30	4.05	4.20
23	4.40	4.20	4.05	4.22
24	4.30	4.15	4.05	4.17
25	4.35	4.20	4.15	4.23
31	4.15	4.05	4.00	4.07
32	4.05	4.20	3.85	4.03
33	4.30	4.25	4.15	4.23
34	4.20	4.25	4.15	4.20
35	4.25	4.10	4.05	4.13
41	4.40	4.30	4.35	4.35
42	4.30	4.40	4.15	4.28
43	4.45	4.35	4.25	4.35
44	4.30	4.20	4.20	4.23
45	4.40	4.30	4.25	4.32
51	3.85	4.25	4.00	4.03
52	4.05	4.35	4.05	4.15
53	4.30	4.25	4.15	4.23
54	3.95	4.05	4.05	4.02
55	4.30	4.25	4.20	4.25
Model Average	4.27	4.26	4.17	4.23

aggregates the average scores ((Clarity + Unity)/2) for each paragraph ID from ChatGPT, DeepSeek, and Qwen evaluations. Overall model averages are computed at the bottom.

4.2.2. Cronbach’s Statistical Consistency Analysis

Cronbach’s alpha ($\alpha$) is a measure of internal consistency reliability, reflecting how consistently multiple items—in this case, the three large language models (ChatGPT, DeepSeek, and Qwen)—assess the same underlying construct (clarity and unity across 25 paragraphs). A high $\alpha$ indicates that the models function as coherent raters, justifying the use of their mean score as a reliable composite metric. Formally, Cronbach’s $\alpha$ is defined as

$$\alpha ={\displaystyle \frac{k}{k-1}}\left(1-{\displaystyle \frac{{\sum }_{i=1}^k{\sigma }_i^2}{{\sigma }_X^2}}\right)={\displaystyle \frac{k·\overline{r}}{1+(k-1)·\overline{r}}}.$$

where $k=3$ is the number of models (items), ${\sigma }_i^2$ is the variance of scores from model i, and ${\sigma }_X^2$ is the variance of the total scores (sum of the three model scores) across paragraphs. The pairwise Pearson correlations between model scores across the 25 paragraphs are shown in

Table 6. Pairwise Pearson correlations between evaluation model scores.

Model Pair	Correlation (r)
ChatGPT vs. DeepSeek	0.43
ChatGPT vs. Qwen	0.52
DeepSeek vs. Qwen	0.46
Average $\overline{r}$	0.47

.

The values yield a mean inter-model correlation of $\overline{r}=0.47$. Substituting into the formula above gives

$$\alpha ={\displaystyle \frac{3\times 0.47}{1+2\times 0.47}}={\displaystyle \frac{1.41}{1.94}}\approx 0.73.$$

The Cronbach’s $\alpha =0.73$ indicates good internal consistency among the three models (ChatGPT, DeepSeek, and Qwen), satisfying the conventional threshold ($\alpha \ge 0.70$) for acceptable reliability in group-level research. Despite minor systematic differences in absolute scoring—such as Qwen’s slightly lower mean (4.17) compared to ChatGPT’s (4.27)—the models exhibit consistent relative judgments, meaning they rank paragraphs similarly in terms of clarity and unity. Because Cronbach’s $\alpha$ assesses the reliability of the composite score rather than absolute agreement, these small calibration biases do not compromise the validity of the aggregate measure. Consequently, the Overall Average column in Table 5 can be treated as a trustworthy and stable proxy for paragraph quality in downstream analyses.

4.2.3. Human Evaluation Summary

To complement automated LLM-based assessments and validate perceptual quality from an end-user perspective, we conducted a small-scale human evaluation study with two participants. Each participant independently rated all 25 generated paragraphs and focus on clarity and unity. The following

Table 7. Human evaluation scores across 25 paragraphs (Clarity + Unity averaged per participant).

Paragraph ID	Participant 1 Average	Participant 2 Average	Human Overall Average
Paragraph 11	4.0	4.5	4.25
Paragraph 12	4.0	4.0	4.00
Paragraph 13	4.5	4.5	4.50
Paragraph 14	4.0	4.0	4.00
Paragraph 15	4.0	4.5	4.25
Paragraph 21	4.5	4.5	4.50
Paragraph 22	4.0	4.0	4.00
Paragraph 23	4.5	4.0	4.25
Paragraph 24	4.0	4.0	4.00
Paragraph 25	4.5	4.0	4.25
Paragraph 31	4.5	4.5	4.50
Paragraph 32	3.5	4.0	3.75
Paragraph 33	4.0	4.5	4.25
Paragraph 34	4.5	4.0	4.25
Paragraph 35	4.0	4.0	4.00
Paragraph 41	4.5	4.5	4.50
Paragraph 42	4.0	4.5	4.25
Paragraph 43	5.0	4.5	4.75
Paragraph 44	4.0	4.0	4.00
Paragraph 45	4.5	4.5	4.50
Paragraph 51	3.5	4.0	3.75
Paragraph 52	4.0	4.0	4.00
Paragraph 53	4.5	4.5	4.50
Paragraph 54	3.5	4.0	3.75
Paragraph 55	4.0	4.5	4.25
Human Average	4.18	4.28	4.23

reports individual scores and their average, providing insight into human judgment consistency.

4.2.4. Overall Assessment

The table below presents a summary of paragraph quality ratings across different generators, as evaluated independently by ChatGPT, DeepSeek, and the Human Average. The (Average) column reports the mean of the two evaluator scores for each generator, while the final row shows the overall evaluator averages. Notably, the MDP Agent is the only system run locally on CPU-only hardware, whereas all other generators leveraged cloud-based GPU resources.

Figure 1 visually reports the data summarized in

Table 8. Summary of generator performance (evaluated by ChatGPT, DeepSeek, and Human Raters).

Generator (Paragraphs)	ChatGPT	DeepSeek	Human Average	Average	GPU/CPU
DeepSeek (11–15)	4.33	4.29	4.20	4.27	Cloud with GPU
ChatGPT (21–25)	4.35	4.22	4.20	4.26	Cloud with GPU
MDP Agent (31–35)	4.19	4.17	4.15	4.17	Local with CPU Only
Copilot (41–45)	4.37	4.31	4.25	4.31	Cloud with GPU
Grok (51–55)	4.09	4.23	4.12	4.15	Cloud with GPU
Evaluator Average	4.27	4.24	4.18	4.23	-

MDP Agent runs on local CPU (i7, 16GB RAM) only—no GPU or cloud.

. The plot highlights the Generator Average.

4.3. Discussion

4.3.1. Generation Sensitivity Across Topics

The design of generation prompts, the formulation of the reward function, and the choice of LLM inference parameters (e.g., temperature, top-p, repetition penalty) significantly influence the quality and consistency of the generated text, and this influence is highly sensitive to the nature of the topic. For technical subjects, dense prompts combined with low temperature ($T=0$) and a clarity-weighted reward ($w_c=0.6$) yielded coherent, accurate explanations. In contrast, for more subjective or narrative topics, the same settings occasionally produced overly rigid prose, suggesting a need for adaptive reward tuning (e.g., higher weight on unity) and slight stochasticity ($T>0$) to preserve stylistic nuance. Our current reward, which relies on LLM-based evaluations of clarity and unity filtered through a fixed sentiment model, works well for structured informational tasks but may under-penalize vagueness or over-penalize creative phrasing in open-ended domains. The MDP Agent’s performance is not uniform across topic categories, and future iterations should incorporate topic-aware reward shaping and dynamic parameter adjustment to balance factual fidelity with expressive flexibility.

4.3.2. Supporting Teachers in Educational Material Design

The integration of AI into teaching practices presents a transformative opportunity to alleviate the cognitive and temporal burdens associated with educational content creation. Our MDP-orchestrated authoring agent is specifically designed to empower educators by enhancing the clarity, and unity of instructional materials through structured, iterative refinement. By leveraging local inference with Qwen3-1.7B-ONNX, the system enables teachers to generate high-quality lesson summaries from complex source texts, such as research papers or technical textbooks, without relying on external cloud services. The MDP framework allows for strategic decision-making in text enhancement, where actions like simplification, elaboration, or tone modulation are applied in a sequence that maximizes pedagogical effectiveness. This capability supports intentional material design, enabling instructors to focus on curricular goals rather than mechanical rewriting. Furthermore, the agent can assist in developing adaptive course content tailored to diverse student populations, including varying reading levels and learning preferences, thereby promoting inclusive and differentiated instruction in higher education.

4.3.3. Enabling Privacy-Preserving AI in Educational Settings

A critical advantage of our approach lies in its on-device deployment via ONNX optimization, which ensures that all text generation and enhancement processes occur locally. This design directly addresses growing concerns about data privacy and security in educational AI applications. Unlike cloud-based large language models that require user inputs to be transmitted to remote servers—potentially exposing sensitive information such as student feedback, assessment comments, or unpublished research—our system keeps all data within the user’s environment. This feature is particularly valuable in institutional contexts governed by strict data protection regulations such as GDPR. It also enables safe deployment in low-bandwidth or restricted-network settings, including schools in resource-constrained regions. By demonstrating that powerful AI assistance can be delivered without compromising user confidentiality, our work contributes to the development of trustworthy, ethically grounded tools that align with the principles of responsible innovation in education.

4.3.4. Promoting Ethical and Trustworthy AI Use

The deployment of AI in education must be accompanied by robust mechanisms to ensure fairness, transparency, and accountability. Our agent addresses these concerns through its interpretable MDP-based orchestration and integrated sentiment analysis module. The decision policy operates over a defined action space—such as simplifying language, enriching explanations, or adjusting tone—allowing educators to understand and audit the system’s behavior. This level of transparency stands in contrast to the opaque, end-to-end outputs of conventional LLMs. Moreover, the reward function within the MDP can be explicitly designed to penalize undesirable traits, including overly complex syntax, biased language, or negative sentiment, thus promoting equitable and constructive communication.

5. Conclusions and Perspectives

This paper has presented an AI authoring agent that leverages the Qwen3-1.7B-ONNX language model and a Markov Decision Process (MDP) framework to enhance the clarity and unity of educational content. By treating text refinement as a sequential decision-making process, the system moves beyond static generation to deliver purposeful, pedagogically guided improvements. Integrated sentiment analysis ensures that outputs maintain a constructive and motivating tone, making the agent particularly suitable for use by teachers, academic writers, and course designers. The local deployment of the LLM via ONNX guarantees data privacy and enables use in secure or low-bandwidth environments—critical advantages for real-world educational settings.

Despite its benefits, the system faces a notable limitation: the iterative nature of the MDP-orchestrated—that runs on CPU only—enhancement process results in a longer total production time compared to single-pass generation models like ChatGPT or DeepSeek. While the initial text generation is fast, each refinement step—guided by policy evaluation, action selection, and quality feedback—adds computational overhead. This latency may affect usability in time-sensitive contexts, such as live classroom preparation or rapid feedback drafting. However, this trade-off reflects a deliberate design choice: sacrificing speed for greater control, transparency, and output quality. The structured enhancement process allows educators to trust the agent’s decisions and understand the rationale behind each edit, aligning with the principles of trustworthy AI in education.

Looking ahead, several directions can reduce this latency and support scalable deployment. First, the MDP policy can be optimized through model distillation or approximate inference to minimize decision-making overhead. Second, parallelization of enhancement actions—such as simultaneous grammar correction and tone adjustment—could reduce sequential steps. Third, adaptive stopping criteria could allow the agent to terminate refinement early when quality thresholds are met, balancing speed and performance. Future work will also explore human-in-the-loop evaluation with educators to measure perceived value versus time cost. As AI becomes more embedded in teaching and learning, systems like ours demonstrate that the future of educational AI lies not in speed alone, but in responsible, interpretable, and pedagogically meaningful augmentation.

To support reproducibility, the source code for the MDP Agent and associated components will be made available upon request by contacting the corresponding author. This approach allows us to share the implementation with appropriate documentation while facilitating direct communication with interested researchers. We hope this hybrid approach —combining locally LLMs with MDP-driven control—can offer a useful reference for future work on authoring agents and educational technology.