Intelligent Tutoring System Based on Local LLMs and RAG ^†

Abstract

This work presents the design and development of an intelligent academic tutoring system aimed at preparing students for higher-education entrance examinations. To validate the effectiveness of the proposal, the university admission examination bank of Ecuador was implemented as a case study. The system integrates three artificial intelligence modules: a model specialized in numerical reasoning adapted through efficient fine-tuning on Qwen 2.5 7B, achieving a loss convergence of 0.37, a functional score of 7.3/10 in qualitative evaluation, and an accuracy of 80.3% on an independent 300-problem evaluation set (99.0% for the base model without fine-tuning, a specialization trade-off analyzed in the paper); a local orchestrator based on large language models (LLMs) with retrieval-augmented generation (RAG); and a linguistic evaluation module aimed at measuring verbal aptitude in Spanish. The methodology encompasses the systemic analysis of the educational domain, the rigorous selection of objective evaluation metrics, and the autonomous deployment of open-source tools on resource-limited hardware. The proposed architecture prioritizes the privacy of student data, the reduction in operational latency, and modular scalability. Taken together, this solution constitutes a comprehensive and secure technological approach to facilitate adaptive teaching processes through fully local educational agents.

1. Introduction

Preparing for university entrance examinations is a significant challenge for applicants, particularly in numerical reasoning and verbal aptitude, where analysis, interpretation, and problem-solving must be performed within limited time frames. Conventional resources (e.g., printed guides, question banks, or generalized courses) often provide repetitive, poorly adaptive practice, limited immediate feedback, and no dynamic adjustment of the study plan. These limitations create an opportunity for intelligent tutoring systems that interact with students, interpret their performance, and provide personalized support [1]. The developed system was validated through a case study based on the admission examination of Escuela Politécnica Nacional (EPN).

Large language models (LLMs) have shown outstanding capabilities in language understanding, guided reasoning, and conversational interaction. These advances rely on Transformer architectures and efficient adaptation techniques such as LoRA (Low-Rank Adaptation) and QLoRA, which enable specialization of pretrained models under hardware constraints [2,3,4]. Complementarily, Retrieval-Augmented Generation (RAG) incorporates external, up-to-date knowledge into the response process, reducing exclusive dependence on learned parameters [5]. However, generalist models do not guarantee optimal performance in specialized educational contexts: tasks such as numerical problems or the evaluation of verbal responses in Spanish require domain adaptation, and academic applications must also consider privacy, latency, autonomy, and feasibility of deployment on local infrastructure.

Accordingly, this work proposes an intelligent academic tutoring system composed of three complementary modules: (i) a model specialized in numerical reasoning and efficiently fine-tuned on domain-aligned problems; (ii) a local LLM orchestrator that routes queries, retrieves contextual knowledge through RAG, and coordinates the specialized models; and (iii) a linguistic evaluation module for verbal aptitude in Spanish, using metrics that combine semantic similarity, grammatical correctness, stability, and computational efficiency. The main contribution is the integration of these capabilities into a modular architecture for the university-admission examination domain. The proposal combines task specialization, local execution, knowledge retrieval, and objective performance evaluation while operating exclusively on private local networks under hardware constraints.

The main contributions of this work can be summarized as follows: (1) a numerical-reasoning tutor built by parameter-efficient QLoRA fine-tuning of Qwen 2.5 7B on a purpose-built dataset of 1945 domain-aligned exercises, together with an explicit, quantified analysis of the specialization trade-off against the base model; (2) a fully local Mixture-of-Experts orchestrator (dual Qwen 2.5 1.5B/14B running on a CPU-only server) whose RAG grounding was improved from a faithfulness of 0.18 to 0.58 (RAGAS-measured, consistent across all four evaluation cases) through task decomposition; (3) a reproducible benchmark of twenty-one open-source language models for verbal-aptitude evaluation in Spanish under a strict 6 GB VRAM constraint; and (4) the end-to-end deployment of the complete tutoring pipeline on institutional infrastructure without any cloud dependence, preserving student data privacy. In contrast to existing LLM-based tutors such as ChatTutor [6] and LPITutor [7], which rely on external cloud APIs and target English-language content, the proposed system operates entirely on local hardware and is specialized for the Spanish-language Ecuadorian admission domain.

The remainder of this paper is organized as follows. Section 2 reviews the background on efficient fine-tuning, RAG, and related intelligent tutoring systems. Section 3 describes the proposed modular architecture and its data flow. Section 4 presents the design and evaluation of the three modules: the numerical reasoning model, the local LLM orchestrator, and the linguistic evaluation module. Section 5 discusses the overall results, the comparison with existing systems, the limitations, and the resource consumption. Section 6 presents the conclusions and future work.

2. State of the Art

2.1. Background

Modern language models are based on the Transformer architecture, which uses attention mechanisms to capture dependencies among the elements of a textual sequence [8]. LLMs are first trained through self-supervised learning over large text collections, acquiring general linguistic knowledge [9]; their performance on specialized tasks such as mathematical reasoning, however, is limited without domain adaptation [10]. Supervised Fine-Tuning (SFT) adapts a pretrained model to a specific task, and Parameter Efficient Fine-Tuning (PEFT) techniques (chiefly LoRA, which injects low-rank matrices into the attention layers, and its quantized extension QLoRA) make this feasible on consumer hardware [2,3]. Specialized mathematical models such as Qwen-Math or DeepSeek-Math exist, but generalist models do not always adapt correctly to specific educational contexts [11]; this work instead fine-tunes a generalist model on data designed for the Ecuadorian admission examination.

Retrieval-Augmented Generation (RAG) combines a generative model with an information-retrieval mechanism over an external knowledge base: for each query it retrieves the most relevant corpus fragments by semantic similarity and adds them as context to the prompt, reducing hallucinations and improving relevance without retraining [5,9]. Dense embeddings and vector databases such as FAISS or ChromaDB are the central components of this pipeline [12].

2.2. Related Work

Intelligent Tutoring Systems (ITSs) aim to provide adaptive, personalized feedback without direct teacher intervention [13,14,15]. Integrating LLMs has expanded their conversational and reasoning capabilities, but most implementations depend on external inference services, introducing privacy, latency, and connectivity constraints that make them unfeasible for public institutions in Latin America. A local Mixture-of-Experts paradigm, where a coordinating model routes queries toward specialized models on the same infrastructure, is a technically viable alternative [16]. Studies such as ChatTutor [17] and LPI-Tutor [18] argue that orchestrating module resources locally increases explanatory support in algorithmic tasks.

Despite the growing body of research on LLMs in education, many architectures are designed primarily for English-language settings or assume extensive VRAM consumption on external servers. These assumptions create significant integration gaps and prohibitive latency for public institutions in Latin America. This work therefore demonstrates that it is methodologically and technically feasible to develop Spanish-language generative tutors for mathematical and verbal reasoning that satisfy admission-test evaluation requirements while operating entirely on private local networks under hardware constraints. A quantitative comparison between the proposed system and representative LLM-based tutoring systems is presented in Section 5.

3. Proposed Solution

The proposed system follows a modular, local-first architecture [19] in which every component runs on the institution’s own infrastructure, with no dependence on external inference services. This decision responds to the three requirements identified in Section 1: the privacy of student data, low and predictable latency, and feasibility of deployment on resource-limited hardware. The architecture is organized around a central orchestrator that coordinates two domain-specialized services (i.e., a numerical reasoning model and a linguistic (verbal) evaluation module) together with a retrieval layer that supplies up-to-date academic knowledge. Figure 1 shows these components and the data flow among them.

Figure 1. Component diagram of the proposed tutoring system: a local orchestrator (application-level Mixture-of-Experts) mediates between the student interface and the domain-specialized services, augments study-plan and feedback decisions with a RAG knowledge base, and returns structured responses over the local private network.

The orchestrator is the entry point of the system and applies an application-level Mixture-of-Experts strategy—that is, a routing pattern among independent models rather than sparse expert layers within a single model. It is exposed as a local REST backend (FastAPI over Uvicorn, port 8000) and relies on a dual-model configuration served through Ollama on a CPU-only server (8 cores, 62 GB RAM): a lightweight model, Qwen 2.5 1.5B, classifies each incoming query, while a larger model, Qwen 2.5 14B (GGUF, Q4_K_M quantization), performs performance analysis and study-plan generation. For every incoming query it identifies the thematic domain and routes the query to the corresponding specialized service; for the study-plan and post-examination feedback flows it additionally retrieves the most relevant fragments from the knowledge base through RAG and injects them into the prompt. Every decision it returns is persisted by the REST backend for auditability and learning-profile construction [20], so that the orchestrator itself remains stateless. Because query understanding, problem-solving, and pedagogical planning are decoupled, each expert can be replaced or retrained independently without altering the rest of the system [21].

Two specialized services sit behind the orchestrator. The numerical reasoning model (Qwen 2.5 7B fine-tuned with QLoRA) runs as an independent inference service through llama.cpp on port 8080 and returns worked solutions, step-by-step explanations, and error diagnoses in JSON. The linguistic evaluation module exposes its own REST endpoints (for example, /generar_pregunta) and measures verbal aptitude in Spanish. The retrieval layer is consumed by the orchestrator rather than by the specialized services: the EPN admission guides and practice examinations are split into 200-word chunks with a 40-word overlap, encoded with the multilingual embedding model paraphrase-multilingual-MiniLM-L12-v2 (384 dimensions), and indexed in a ChromaDB vector store (137 fragments) for cosine-similarity search. This layer grows by simply re-embedding and re-indexing new material, without retraining any model [5,9].

The end-to-end data flow is as follows. The student interacts only with the web interface, designed following established usability and ergonomics criteria [16,17,18], which sends each query to the orchestrator over HTTP in JSON format. The orchestrator classifies the query and forwards it to the numerical service or the verbal module as appropriate; when the request corresponds to a study-plan or feedback decision, it first augments the prompt with the context retrieved from ChromaDB. The selected service returns a structured response that the orchestrator post-processes and delivers back through the backend, which persists it in the learning profile and returns it to the interface. All communication travels over the local private network, so no student data leaves the institutional infrastructure at any point.

4. Experimentation and Testing

4.1. Numerical Model (EPN Case)

The numerical reasoning model was developed through supervised fine-tuning over four phases: domain analysis, dataset construction, training, and comparative evaluation. The model targets numerical reasoning problems from the entrance examination to Ecuadorian universities; the main exercise types are summarized in Table 1.

Table 1. Classification of Numerical Reasoning Problems.

Problem Type	Skill Assessed
Numerical sequences	Pattern identification
Rule of three	Direct and inverse proportionality
Verbal algebra	Translating language into equations
Age problems	Algebraic relationships
Work problems	Time-productivity relationships
Logic problems	Quantitative inference

A supervised dataset of 1945 examples was built, focused on the syllabus of entrance tests (rule of three, percentages, sequences, etc.), structured in the Alpaca format with explicit instruction modes (e.g., MODE: EVALUATE_ANSWER). A response-only loss masking approach was used so that instruction and input tokens do not affect the gradient update (labels = −100). The dataset, generated synthetically with Gemini 2.5 Flash, is distributed as shown in Table 2. The corpus was built iteratively: an initial set of 1638 examples was expanded with 307 additional examples with explicit result verification, targeting the weakest instruction modes, which yielded the final distribution shown in Table 2 and the last fine-tuning iteration (v5).

Table 2. Dataset Distribution by Instruction Mode.

#	Mode	Examples	%
1	Free generation	958	49.3%
2	Explain concept	293	15.1%
3	Evaluate answer	217	11.2%
4	Correct evaluation	211	10.8%
5	Generate by topic	266	13.7%
	Total	1945	100%

The base model was Qwen 2.5 7B Instruct (Table 3), chosen for its superior mathematical performance in its class and its Spanish support, resolving the hardware trade-off through QLoRA quantization. Training used PEFT (LoRA) with 4-bit NF4 QLoRA quantization through HuggingFace Transformers; the final hyperparameters are listed in Table 3. Covering the four attention matrices (rather than only q_proj and v_proj) was required for stable learning. Over 976 iterations the loss converged stably from 1.39 to 0.37 in approximately 4 h 4 min.

Table 3. Base Model and Training Hyperparameters (LoRA/QLoRA).

Base Model	Value	Hyperparameter	Value
Model	Qwen 2.5 7B Instruct	Target modules	q_proj, k_proj, v_proj, o_proj
Parameters	7.6B	LoRA rank (r)/α	16/32
Architecture	Transformer (GQA)	Learning rate	1.5 × 10−4
Context length	128k tokens	Batch/grad. accum.	1/8
Training type	Instruct tuned	Epochs (steps)	4 (976)
Quantization	4-bit NF4 (Double)	Optimizer	paged_adamw_8bit

The model was evaluated on an independent set of 300 multiple-choice problems not seen during training, automatically generated with programmatically verified answer keys, following the same output structure as the training data (statement, four options, verified answer, and step-by-step solution) and covering the problem types of Table 1 in ten balanced families. Performance is measured with accuracy:

Accuracy = (Correct Answers)/(Total Questions)

(1)

A response is counted as correct only when the selected option exactly matches the verified key. Under this protocol, the fine-tuned model (v5) reaches an accuracy of 80.3% (241/300), whereas the base Qwen 2.5 7B, evaluated on the same problems, reaches 99.0% (297/300) (Table 4). The 95% Wilson confidence intervals for these proportions are 75.5–84.4% (fine-tuned) and 97.1–99.7% (base). Since both models were evaluated on the same 300 items and the base model fails at most 3 of them, a McNemar analysis of the paired responses yields p < 0.001 for any feasible split of the discordant pairs, so the observed difference is statistically significant. Because each of the ten problem families contributes the same number of items, the evaluation set is balanced by construction and accuracy is not distorted by class imbalance; the value-level accuracy reported below (82.0% vs. 97.7%), which removes the option-selection bias, complements the raw multiple-choice figure. A per-response analysis shows that the gap has two sources: an option-selection bias (in 14 cases v5 computes the correct value but selects the wrong option; scored on the final numeric value, v5 reaches 82.0% versus 97.7% for the base) and arithmetic errors introduced by fine-tuning on synthetic pedagogical data. The domain adaptation therefore specializes in the model for its tutoring modes at a measurable cost in raw multiple-choice accuracy, a trade-off consistent with the forgetting effects reported for narrow-domain supervised fine-tuning. Complementarily, a qualitative functional evaluation of the six tutoring modes was performed across the fine-tuned versions (Table 5): the final model (v5) reaches an average score of 7.3/10, a 97% improvement over the fine-tuned Llama 3.2 3B baseline (v3, 3.7/10) and 12% over the previous iteration on the same base model (v4, 6.5/10). Figure 2 confirms the continuous execution of the numerical model via llama.cpp on the local EPN infrastructure.

Table 4. Accuracy Comparison on Mathematical Benchmarks.

Model	GSM8K (%)	MATH (%)	EPN-300 (%)
Llama 3.1 8B Instruct	78.1	51.9	—
Qwen 2.5 7B (base)	91.6	75.5	99.0
Qwen 2.5 7B FT	—	—	80.3

Table 5. Qualitative Functional Evaluation of the Six Tutoring Modes across Fine-Tuned Versions (score out of 10).

Tutoring Mode	v3 (Llama 3.2 3B)	v4 (Qwen 2.5 7B)	v5 (Final)
1—Free generation	4/10	7/10	9/10
2—Generate by topic	5/10	3/10	7/10
3—Explain concept	3/10	2/10	5/10
4—Evaluate correct answer	2/10	10/10	8/10
5—Evaluate incorrect answer	4/10	8/10	8/10
6—Correct evaluation	4/10	9/10	7/10
Average	3.7/10	6.5/10	7.3/10

Figure 2. Status of the llama-tutor service (systemctl) with model_tutor_q4km.gguf active on the dtic-system server, listening on port 8080.

4.2. Local LLM Orchestration Module

The orchestrator coordinates the flow of information among the student interface, the domain-specialized models, and the performance analysis system, following a Mixture-of-Experts approach: it (1) classifies the thematic domain of each query with a local LLM, (2) routes the problem to the corresponding specialized model, (3) generates adaptive study paths prioritizing the topics of lowest mastery, as well as post-examination feedback, both grounded in contextual knowledge retrieved through RAG, and (4) returns every decision together with its rationale and model version, which the backend persists for auditability.

Five open-source candidates (Qwen 2.5 1.5B, Llama 3.2 3B, Phi-3.5 [19], Mistral 7B, Qwen 2.5 14B) were benchmarked with a role-specific evaluation designed around the three functional responsibilities of the module: routing accuracy over 20 balanced labeled queries (Block 1), a four-criterion binary rubric for performance-profile analysis over three synthetic student profiles (Block 2), and a five-criterion pedagogical rubric for study-path generation (Block 3). Raw Block 1 accuracy ranged from 0.55 to 0.60 across all models; error analysis showed that the misses were caused by typographic output variants (accents, plurals) rather than by misclassification, so the production pipeline adds an eight-example few-shot prompt with explicit category definitions together with a Unicode NFD normalization stage. The deployed classifier was subsequently re-evaluated on an expanded set of 100 balanced queries (50 numerical, of which 12 are spatial-aptitude items, and 50 verbal) covering the ten subject areas of the EPN examination and written in the colloquial register of an actual applicant; the eight prompt examples were excluded from the test set. Under this protocol Qwen 2.5 1.5B reaches an accuracy of 0.88 (88/100), with a 95% Wilson confidence interval of 0.80–0.93 and per-class F1 scores of 0.870 for numerical and 0.889 for verbal queries. All 12 spatial-aptitude items were routed correctly, which empirically supports the decision to group spatial reasoning under the numerical category. Notably, the constrained prompt eliminated typographic variants altogether—every one of the 100 responses was returned as an exact category token—so raw and normalized accuracy coincide and NFD normalization operates as a safeguard rather than as the mechanism responsible for the improvement. The residual errors are concentrated in definition-seeking phrasings of numerical topics (for example, questions opening with ‘what is’ or ‘explain’), which the classifier tends to route to the verbal specialist: ten of the twelve errors follow this pattern, and per-subject accuracy falls to 10/16 in arithmetic and 5/8 in geometry while remaining perfect in algebra and in every verbal category except analogies (8/10). Qwen 2.5 14B achieved the maximum score in Block 2 (4.0/4), tied with Mistral 7B, and in Block 3 all five models reached the maximum score (5.0/5); it was selected over Mistral for its greater consistency in producing valid structured JSON output. A dual-model architecture was therefore adopted: Qwen 2.5 1.5B handles classification (synchronous, ~0.9–13 s), while Qwen 2.5 14B handles analysis and planning (asynchronous tasks executed at the end of a session or examination).

The RAG pipeline segments the EPN admission materials into 200-word chunks with a 40-word overlap. Three of the six source documents lacked sufficient native text and were processed with Tesseract OCR (Spanish language model, 250 dpi); after removing platform boilerplate and discarding low-legibility fragments (heuristic score below 0.55, based on alphabetic-character and recognizable-word ratios), 64 OCR-derived fragments were retained out of 87 evaluated, at a consistent mean score of ≈0.60 across the three documents. Combined with 73 native-text fragments, the resulting index holds 137 fragments, encoded with paraphrase-multilingual-MiniLM-L12-v2, and stores them in ChromaDB; at inference time, the two fragments most similar to the student’s weak topics (mastery < 0.70) are truncated to 800 characters and injected into the prompt.

Response quality was assessed with a structured qualitative evaluation following the RAGAS criteria (faithfulness, answer relevancy, context precision, context recall) over four student profiles tied to real EPN examination areas. The baseline showed high answer relevancy (≈0.89) but limited faithfulness, as rationales paraphrased the structured input instead of citing the retrieved EPN material. After three prompt-engineering iterations failed to improve grounding, the generation task was decomposed into two steps—plan selection followed by an isolated rationale-writing call receiving only the retrieved context. Automated evaluation with the RAGAS framework (qwen2.5:14b as local LLM-judge, over the 137-fragment index) confirmed the improvement across all four evaluation cases, raising mean faithfulness from 0.18 to 0.58 without affecting answer relevancy (≈0.75) or context precision/recall (≈0.50/≈0.55), which remained stable between conditions as expected given the identical retrieved context. Table 6 summarizes the results; end-to-end latencies were 246–265 s for asynchronous plan generation. Figure 3 shows the local FastAPI backend and its/health verification.

Table 6. Evaluation of the Local LLM Orchestrator.

Metric	Result
Classifier/analyzer models	Qwen 2.5 1.5B/Qwen 2.5 14B (Ollama, Q4_K_M, CPU)
Embeddings/vector DB	paraphrase-multilingual-MiniLM-L12-v2/ChromaDB (137 chunks)
Routing accuracy (Block 1 benchmark, 20 queries/deployed classifier, 100 queries)	0.55–0.60/0.88 (95% CI 0.80–0.93)
Analysis rubric (4 criteria)/path rubric (5 criteria)	4.0/4–5.0/5
Answer relevancy/context precision (RAGAS criteria)	≈0.75/≈0.50 (RAGAS-measured, qwen2.5:14b judge)
Faithfulness (RAGAS, qwen2.5:14b judge): generic/decomposed	0.18/0.58
Latency: classification/plan generation with RAG	≈0.9–13 s/246–265 s

Figure 3. End-to-end sequence of the local LLM orchestrator, from the authenticated student request to the persistence of the academic decision. Synchronous query routing (top alternative) is resolved by the lightweight classifier without retrieval, whereas study-plan and feedback decisions (bottom alternative) are grounded in RAG context and executed asynchronously. Measured latencies are shown per interaction.

4.3. Linguistic Evaluation Module—NLP

This module addresses the selection of a language model for linguistic correction and educational feedback (a task related to Grammatical Error Correction (GEC) [20]) under a strict hardware constraint (an NVIDIA RTX 3060 Laptop GPU with 6.0 GB). Following the recommendation that LLM evaluation requires robust, multidimensional frameworks rather than a single metric [8,21], the model was assessed on output quality, linguistic correction, consistency, and technical feasibility. Six metrics were used: perplexity (fluency), reference similarity (BLEU/ROUGE/BERTScore [22,23,24], which are strongly correlated [25]), grammatical error reduction, semantic coherence, consistency across iterations, and computational efficiency. The key formulas are:

$$Perplexity= exp({\displaystyle \frac{1}{N}} {\displaystyle \sum _{i=1}^N}{L}_i)$$

(2)

$${∆}_{errors}={\displaystyle \frac{E_{before}-E_{after}}{E_{before}}},\hspace{1em}Consistency={\displaystyle \frac{1}{K}}\sum _{i<j}{s}_{ij, }\hspace{1em}{\overline{t}}_{inf}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{t}_i$$

(3)

where L_i is the per-example loss, E_before/after the error counts before/after correction, s_ij the cosine similarity between two outputs for the same input over K pairs, and t_i the per-sample generation time. The evaluation was implemented through an orchestrator that detects available hardware, verifies VRAM compatibility, runs each metric in a decoupled module, and consolidates results over a JSON benchmark of input–reference pairs. The per-candidate screening (Table 7) used a deliberately small probe set (2 input–reference pairs) as a lightweight comparative filter, and the selection was confirmed on the larger 120-pair set used for the fine-tuning A/B evaluation (derived from the held-out test split of the verbal-aptitude dataset). Both sets originate from the project dataset, generated with assistant LLMs and manually validated (see the Use of Artificial Intelligence statement). The twenty-one candidate models are listed in Table 7, and Figure 4 shows the question-generation endpoint in operation.

Table 7. Evaluation results of the 21 candidate language models under the 6 GB hardware constraint (RTX 3060). Δerrors is the relative grammatical-error reduction on the screening probe (E_before − E_after)/E_before (↑ better: 1.00 = all errors removed, 0.00 = no net change, <0 = errors introduced); it is a secondary screening signal, not a selection criterion. In the column headers, ↓ indicates that lower is better and ↑ that higher is better.

Model	Runs	Perplexity ↓	BERTScore-F1 ↑	Coherence ↑	Consistency ↑	Δerrors ↑	Inf. Time (s) ↓	VRAM (GB)
Phi-3-mini-4k-Instruct (3.8B)	Yes	11.73	0.732	0.675	0.915	0.50	20.1	4.96
Qwen2.5-7B-Instruct	Yes	11.85	0.717	0.339	0.606	0.50	4702.2	5.23
Nous-Hermes-2-Mixtral-8x7B	Yes	16.73	0.649	0.354	0.607	0.50	1333.4	4.65
Falcon-7B-Instruct	Yes	21.81	0.802	0.842	0.591	1.00	329.6	4.57
SOLAR-10.7B-Instruct	Yes	22.16	0.748	0.819	0.883	1.00	922.5	5.21
Llama-3.1-8B-Instruct	Yes	25.25	0.716	0.784	0.710	0.50	496.6	4.96
Llama-3.2-3B-Instruct (selected)	Yes	31.86	0.803	0.721	0.899	0.00	30.7	4.41
Zephyr-7B-Alpha	Yes	32.56	0.688	0.806	0.607	0.50	362.3	4.41
Mistral-7B-Instruct-v0.2	Yes	39.31	0.732	0.627	0.650	0.50	5528.9	4.41
StableLM-Zephyr-3B	Yes	47.16	0.732	0.510	0.530	1.00	13.3	4.64
Kukedlc-Llama-7B-Spanish	Yes	52.39	0.726	0.819	0.730	1.00	327.0	5.28
TinyLlama-1.1B-Chat	Yes	63.06	0.721	0.819	0.693	1.00	2.2	2.06
Gemma-2-9B-it	Yes	67.14	0.631	0.090	0.548	1.00	5699.2	4.14
Aya-23-8B	Yes	109.03	0.728	0.797	0.602	1.00	464.1	4.42
GPT2-Spanish	Yes	124.63	0.610	0.661	0.675	−1.50	3.1	5.28
BERTIN-GPT-J-6B	No	—	—	—	—	—	—	—
DeepSeek-Coder-6.7B	No	—	—	—	—	—	—	—
Galactica-6.7B	No	—	—	—	—	—	—	—
Phi-3-Small-128k	No	—	—	—	—	—	—	—
OpenLLaMA-7B	No	—	—	—	—	—	—	—
Phi-3-Small-7B-GPTQ-4bit	No	—	—	—	—	—	—	—

Figure 4. Functional test of the/generar_pregunta endpoint: the request sets topic, difficulty, and number of options, and the service returns a JSON structure with the generated question, alternatives, and correct answer, validating the REST API–LLM integration.

The implementation established a reproducible, multidimensional evaluation methodology and confirmed that the hardware constraint directly affects model selection: of the twenty-one candidates, fifteen ran within the 6 GB budget while six could not be loaded, and several viable models required partial offloading to CPU/disk with prohibitive latency. Table 7 reports, per model, perplexity, semantic similarity (BERTScore-F1), semantic coherence, cross-run consistency, grammatical-error reduction (delta-errors), inference time, and peak VRAM. Llama-3.2-3B was selected: it achieved the highest BERTScore-F1 (0.803) and the second-highest cross-run consistency (0.899, after Phi-3-mini-4k-Instruct, 0.915), combined with higher semantic coherence than Phi-3-mini (0.721 vs. 0.675) and a competitive inference time (30.7 s), together with native Spanish support and a mature QLoRA/GGUF ecosystem. Lexical-overlap metrics (BLEU/ROUGE) were not used per candidate, since they reward literal n-gram matches and penalize the valid paraphrases typical of open-ended feedback; BERTScore was therefore adopted as the semantic reference-similarity metric. This is consistent with the fine-tuning A/B evaluation of the selected model, where lexical-overlap scores were very low (BLEU~0.0, ROUGE-L~0.06, METEOR [26]~0.02) while the semantic metrics remained informative.

Fine-tuning of the selected model. Llama-3.2-3B was subsequently specialized to the verbal-aptitude domain through QLoRA (4-bit NF4 quantization with low-rank LoRA adapters) using a staged schedule on the same RTX 3060 (6 GB), for a total of about 3 h 15 min. An A/B evaluation over 120 prompts (base vs. fine-tuned) confirmed consistent gains in perplexity (28.7 → 13.6), semantic coherence (0.66 → 0.74) and cross-run consistency (0.81 → 0.91), with a modest reduction in exact-content accuracy (0.70 → 0.59) and an essentially stable composite score (0.719 → 0.728). A paired Wilcoxon signed-rank test on the 120 per-item values confirmed that the perplexity and coherence gains are statistically significant (perplexity median 28.4 → 13.0, p < 0.001; coherence median 0.67 → 0.77, p < 0.001; both models scored under identical 4-bit NF4 inference, isolating the effect of the LoRA adapter). This is the A/B referenced above, and the resulting adapter is the one served by the deployed/generar_pregunta endpoint (Figure 4).

5. Results and Discussion

Table 8 synthesizes the three AI modules. The numerical module shows the highest relative performance: the Qwen 2.5 7B (v5) model converges to a loss of 0.37 in 976 steps over 1945 examples, from a base that already leads its class (GSM8K 91.6%, MATH 75.5%). The local orchestrator (dual Qwen 2.5 1.5B/14B via Ollama) achieved maximum scores on the role-specific rubrics (4.0/4 analysis, 5.0/5 path generation), an answer relevancy of ≈0.75, and a RAGAS-measured faithfulness of 0.58 after decomposing the generation task, with synchronous classification in ~0.9–13 s and asynchronous RAG-augmented planning in 246–265 s, confirming the viability of a fully local orchestrator on CPU-only infrastructure for low-connectivity environments [5,9]. The NLP module, on 6 GB of VRAM, compared 21 models with six metrics, prioritizing the balance between semantic quality (BERTScore) and operational viability [25].

Table 8. Comparative Summary of the System’s AI Modules.

Aspect	Numerical	LLM Orch.	Verbal NLP
Base model	Qwen 2.5 (7B)	Qwen 2.5 (1.5B/14B)	Best of 21
Adaptation	QLoRA/LoRA	Few-shot + RAG + Q4_K_M	Fine-tuning
Main metric	Accuracy (300 set)	Faithfulness (RAGAS criteria)	BERTScore F1
Result	80.3%	0.58	0.803
Latency	<1 s	0.9–13 s (sync)/246–265 s (async)	7.1 s
VRAM (inf.)	≈4.5 GB	CPU, ≈10 GB RAM	≤6 GB
Cloud req.	No	No	No

To position the proposal against existing LLM-based tutoring systems, Table 9 contrasts the proposed numerical tutor with LPITutor [13] and ChatTutor [12]. LPITutor, built on GPT-3.5 through a cloud API with RAG over computer-science lecture material, reports response accuracies between 89% (advanced questions) and 97% (beginner questions) and a user satisfaction of 3.8/5; ChatTutor, based on chained cloud LLMs, reports user-study ratings between 4.2 and 4.9 on a 5-point scale and a stability score of 0.68 versus 0.49–0.56 for its ablated variants. With an accuracy of 80.3% on its 300-problem domain set and a functional score of 7.3/10, the proposed system remains below the accuracy range reported by LPITutor, but it is the only one of the three that runs fully on local infrastructure, operates in Spanish, and is specialized in the mathematical domain of the Ecuadorian admission examination, with sub-second inference latency on a mid-range GPU.

Table 9. Quantitative comparison with existing LLM-based tutoring systems.

Aspect	LPITutor [13]	ChatTutor [12]	This Work
Base LLM	GPT-3.5 (cloud API)	ChatGPT/GPT-4 (cloud API)	Qwen 2.5 7B FT (local)
Deployment	Cloud	Cloud	Local (llama.cpp)
Domain/language	CS courses/English	Open topics/English	Admission math/Spanish
Adaptation	RAG + prompting	LLM chaining	QLoRA FT + RAG
Reported accuracy	89–97%	n/r	80.3%
User-facing quality	3.8/5 satisfaction	4.2–4.9/5 user study	7.3/10 functional
Cloud dependence	Yes	Yes	No

Limitations. The numerical training set comes exclusively from the EPN bank, which may cause overfitting on unseen problem structures, and accuracy was measured over a moderate set of 300 instances [26]. In addition, the accuracy of the fine-tuned model (80.3%) is below that of its base model (99.0%): fine-tuning on synthetic pedagogical data introduced arithmetic regressions and an option-selection bias, so future dataset iterations will require professionally verified mathematics. Likewise, the fine-tuned model was not re-evaluated on the public mathematical benchmarks (GSM8K, MATH) owing to hardware availability constraints; quantifying the specialization trade-off on these benchmarks, with base and fine-tuned models under identical quantization settings, is left as future work. The routing classifier, evaluated on 100 labeled queries, exhibits a systematic bias towards the verbal specialist when numerical topics are phrased as requests for a definition rather than for a procedure, which accounts for ten of its twelve errors and suggests that the few-shot examples should cover both phrasings. The orchestrator depends on the coverage of the 77 ChromaDB fragments after OCR processing of the three source documents lacking native text; roughly a quarter of the OCR-extracted fragments were discarded for low legibility, and out-of-corpus queries degrade context precision to ≈0.17—while the RAGAS-based scores come from a structured qualitative assessment rather than the automated LLM-as-judge framework, and rationale quality remains bounded by retrieval precision when noisy fragments are recovered. The verbal NLP module is constrained by the scarcity of quality-annotated Spanish corpora for the Ecuadorian domain; lexical-overlap metrics (BLEU, ROUGE) underestimate semantic quality and must be read together with BERTScore and coherence [25,27].

Resources and scalability. As summarized in Table 10, only the numerical module required prolonged training (4 h 4 min on an RTX 3060 12 GB) and the NLP module a shorter fine-tuning stage (3 h 15 min on an RTX 3060 6 GB), whereas the orchestrator operates purely in inference mode, and the orchestrator runs entirely on CPU (8 cores), requiring ~2 GB of RAM for classification and ~10 GB for analysis and planning. The complete system thus deploys on modest institutional hardware, requiring only a mid-range GPU (6–12 GB) for the specialized models and a CPU-only server for the orchestrator, without cloud dependencies [7,12]. The architecture scales horizontally by registering new specialized models in the orchestrator and vertically by expanding the RAG corpus without retraining (only re-embedding and indexing) [5,9]. The main scalability concern is the accumulated end-to-end latency, which may require parallelism or load balancing under many simultaneous students [27].

Table 10. Computational Resource Consumption per Module.

Resource	Numerical	LLM/RAG	Verbal NLP
Training GPU	RTX 3060 12 GB	N/A	RTX 3060 6 GB
Inference VRAM	≈4.5 GB	CPU, ≈10 GB RAM	≤6 GB
Training time	4 h 4 min	N/A	3 h 15 min
Latency	<1 s	0.9–13 s/246–265 s (async)	7.1 s
Cloud req.	No	No	No

6. Conclusions

This work presented the architecture, methodology, and evaluation of an intelligent academic tutoring system for university entrance preparation, deployed entirely on local infrastructure. The partial results confirm the viability of efficient fine-tuning (LoRA/QLoRA) to specialize LLMs in numerical reasoning on limited hardware—reaching 80.3% accuracy on an independent 300-problem evaluation set, with a measured specialization trade-off relative to the base model (99.0%) the effectiveness of RAG in supplying specific academic knowledge to a local LLM without retraining, the relevance of hybrid metrics (BLEU, ROUGE, METEOR, BERTScore) for evaluating verbal aptitude in Spanish, and the robustness of the modular backend as a secure orchestrator. Future work includes collecting real interaction data for continuous retraining, expanding the RAG corpus, a longitudinal evaluation of pedagogical impact, and exploring multimodal models for content with graphs and diagrams.