Development of an Automotive Electronics Internship Assistance System Using a Fine-Tuned Llama 3 Large Language Model

Abstract

This study develops and validates an artificial intelligence (AI)-assisted internship learning platform for automotive electronics based on the Llama 3 large language model, aiming to enhance pedagogical effectiveness within vocational training contexts. Addressing critical issues such as the persistent theory–practice gap and limited innovation capability prevalent in existing curricula, we leverage the natural language processing (NLP) capabilities of Llama 3 through fine-tuning based on transfer learning to establish a specialized knowledge base encompassing fundamental circuit principles and fault diagnosis protocols. The implementation employs the Hugging Face Transformers library with optimized hyperparameters, including a learning rate of 5 × 10⁻⁵ across five training epochs. Post-training evaluations revealed an accuracy of 89.7% on validation tasks (representing a 12.4% improvement over the baseline model), a semantic comprehension precision of 92.3% in technical question-and-answer assessments, a mathematical computation accuracy of 78.4% (highlighting this as a current limitation), and a latency of 6.3 s under peak operational workloads (indicating a system bottleneck). Although direct trials involving students were deliberately avoided, the platform’s technical feasibility was validated through multidimensional benchmarking against established models (BERT-base and GPT-2), confirming superior domain adaptability (F1 = 0.87) and enhanced error tolerance (σ² = 1.2). Notable limitations emerged in numerical reasoning tasks (Cohen’s d = 1.15 compared to human experts) and in real-time responsiveness deterioration when exceeding 50 concurrent users. The study concludes that Llama 3 demonstrates considerable promise for automotive electronics skills development. Proposed future enhancements include integrating symbolic AI modules to improve computational reliability, implementing Kubernetes-based load balancing to ensure latency below 2 s at scale, and conducting longitudinal pedagogical validation studies with trainees. This research provides a robust technical foundation for AI-driven vocational education, especially suited to mechatronics fields that require close integration between theoretical knowledge and practical troubleshooting skills.

1. Introduction

As the automotive industry undergoes rapid transformation driven by digitalization, electrification, and artificial intelligence, vocational education programs have encountered growing challenges in meeting evolving industrial demands [1,2]. Conventional curricula in automotive electronics, while well-established in teaching fundamentals such as lighting systems and ignition circuits, often fall short in preparing students for real-world problem-solving involving cyber–physical systems and AI-assisted diagnostics [1].

The persistent theory–practice gap is not merely pedagogical but structural. Instruction often relies on rigid content delivery, limiting student engagement and adaptive thinking. Despite pedagogical advances such as flipped classrooms and project-based learning, vocational education still struggles to incorporate interdisciplinary competencies, particularly those associated with emerging intelligent systems [3].

Large language models (LLMs), particularly Meta’s Llama 3, present promising opportunities to enhance instructional design through contextualized, human-like dialogue and domain-specific adaptability [4,5]. Unlike traditional intelligent tutoring systems (ITSs), which are limited in scope and responsiveness, Llama 3 is capable of continuous learning through transfer learning, enabling scalable support for dynamic technical education needs [6,7,8].

Integrating AI into internship learning platforms addresses multiple levels of pedagogical improvement: (1) reducing barriers to expert knowledge through natural language Q&A, (2) simulating fault scenarios in virtual environments for circuit analysis, and (3) enabling real-time feedback and assessment. Such platforms also support the scalability and personalization necessary for diverse student cohorts with varying proficiency levels [9].

However, while recent advancements in AI application in education have demonstrated improvements in conceptual understanding and task automation [10], domain-specific implementation—particularly in the field of mechatronics—remains underexplored. The existing literature primarily emphasizes general education or computer science applications, leaving a gap in vocational and technical training contexts.

This study seeks to fill that gap by designing and validating an AI-enhanced internship learning system built upon a fine-tuned Llama 3 model, offering a novel framework for vocational training in automotive electronics through domain-specific adaptation and scalable architecture.

By constructing a robust domain knowledge base through systematic data collection and processing, optimizing platform architecture with integrated layers, and benchmarking technical (e.g., 92.3% semantic accuracy) and pedagogical outcomes, this research proposes a replicable framework for intelligent vocational training systems, particularly in automotive electronics.

2. Literature Review

2.1. Applications of Artificial Intelligence in Education

Artificial intelligence in education encompasses a wide range of applications, as summarized in Table 1. These applications are categorized into five key domains: intelligent teaching systems, adaptive learning platforms, virtual reality teaching, edge-cloud collaborative technology, and intelligent tutoring systems. Each domain demonstrates distinct technical characteristics and educational benefits.

Intelligent teaching systems leverage learner data and instant feedback mechanisms to deliver personalized content and optimize learning outcomes [11,12,13,14,15,16]. Recent studies highlight AI’s role in vocational education, particularly in technical fields like automotive electronics, where adaptive platforms enhance skill acquisition [16,17,18,19,20].

Adaptive learning platforms focus on dynamically modifying instructional difficulty and content delivery based on learner profiles. Coursera’s application of machine learning algorithms and Smart Sparrow’s multimodal interactive physics modules illustrate how such platforms provide differentiated learning paths that adapt to individual needs [9,21].

Virtual reality (VR) technology has been increasingly recognized for its role in teacher training programs, providing immersive experiences through simulated environments that enable teachers to practice teaching skills without real-world consequences. Stavroulia, Baka, and Lanitis (2025) [22] investigated the design of VR-based teacher training environments, with a particular focus on the appearance of virtual classrooms. Their findings indicate that teachers prefer imaginative and fictional virtual classroom settings over standard, realistic classroom environments. These imaginative settings were found to enhance the sense of presence and engagement, as evidenced by increased user participation in activities and electroencephalogram (EEG) data demonstrating improved visual attention [22].

Edge-cloud collaborative technology enhances system responsiveness and scalability by integrating localized device processing with cloud-based computation. This architecture facilitates real-time educational experiences while supporting complex AI operations in decentralized environments, ensuring efficient resource allocation [21].

Intelligent Tutoring Systems integrate natural language processing and emotion recognition to deliver real-time, emotionally responsive instruction. Duolingo’s adaptive algorithm for language learning and MATHia’s math diagnostics platform exemplify this approach, offering highly customized and emotionally attuned learning assistance [10].

While these developments represent significant progress in digital pedagogy, they are predominantly implemented in general education contexts. Their adaptation for vocational and domain-specific training remains limited. As such, this study responds to the need for tailored, scalable AI applications in technical education by leveraging LLMs such as Meta’s Llama 3, capable of integrating domain-specific knowledge with dynamic instructional design.

Table 1. Cases of artificial intelligence application in education.

Application Direction	Technical Features	Reference Source
Intelligent Teaching Systems	Personalized recommendations based on learner data, instant feedback mechanisms	Chuanxiang Song (2024) [11], Chopra, D. (2023) [9]
Adaptive Learning Platforms	Dynamic adjustment of content difficulty, multimodal interaction design	Chuanxiang Song (2024) [11], Chopra, D. (2023) [9], Zhang (2024) [21]
Virtual Reality Teaching	3D environment simulation, multisensory immersive experience	Stavroulia et al. (2024) [22]
Edge-Cloud Collaborative Technology	Intelligent allocation of local devices and cloud computing resources	Zhang (2024) [21]
Intelligent Tutoring Systems	Natural language processing, emotion recognition	Coelho (2023) [10]

Table 1. Cases of artificial intelligence application in education.

Application Direction	Technical Features	Reference Source
Intelligent Teaching Systems	Personalized recommendations based on learner data, instant feedback mechanisms	Chuanxiang Song (2024) [11], Chopra, D. (2023) [9]
Adaptive Learning Platforms	Dynamic adjustment of content difficulty, multimodal interaction design	Chuanxiang Song (2024) [11], Chopra, D. (2023) [9], Zhang (2024) [21]
Virtual Reality Teaching	3D environment simulation, multisensory immersive experience	Stavroulia et al. (2024) [22]
Edge-Cloud Collaborative Technology	Intelligent allocation of local devices and cloud computing resources	Zhang (2024) [21]
Intelligent Tutoring Systems	Natural language processing, emotion recognition	Coelho (2023) [10]

2.2. Meta Llama 3 and Domain Adaptation

Meta Llama 3, a state-of-the-art large language model developed by Meta AI, offers significant potential for educational applications, particularly in domain-specific learning environments. Built upon the Transformer architecture, Llama 3 introduces grouped-query attention mechanisms and extended context windows, optimizing both memory efficiency and inference accuracy. The model is released in 8B and 70B parameter configurations and trained on public datasets containing trillions of tokens.

Unlike general-purpose language models, Llama 3 is designed for high adaptability in specialized domains, with recent studies demonstrating its efficacy in fine-tuning for technical education and automotive diagnostics [4,5,17,23,24]. It supports transfer learning and fine-tuning techniques that allow educators and developers to embed domain-specific corpora—such as automotive repair manuals, technical specifications, and instructional protocols—into its neural structure. This flexibility makes Llama 3 particularly well-suited for applications requiring technical vocabulary comprehension, procedural reasoning, and contextual diagnostics.

In multilingual and domain-sensitive tasks, Llama 3 has demonstrated robust generalization and resistance to prompt perturbation. For example, evaluations from German crowdsourced language assessments report improved robustness and semantic understanding compared to earlier models. Despite these advantages, limitations persist. Performance in numerical reasoning and sequential logic tasks remains inferior to that of domain experts, suggesting that integration with symbolic AI or rule-based logic engines may be necessary for certain use cases.

In educational contexts, Llama 3’s capabilities extend beyond static content generation. It enables interactive, dialogue-based instruction, real-time feedback loops, and adaptive questioning, all of which are essential for learner engagement in technical fields. When fine-tuned on vocational data—such as automotive electronics diagnostics—Llama 3 can simulate expert-level guidance, offering step-by-step troubleshooting, decision tree navigation, and safety compliance validation.

The scalability of Llama 3 also supports deployment in cloud-based educational infrastructures. With appropriate optimization, its inference latency and resource utilization can be tailored to classroom or remote settings, allowing institutions to integrate AI without incurring prohibitive computational costs. Thus, Meta Llama 3 serves not only as a content generator but as an intelligent scaffold for building customized educational ecosystems aligned with industry demands.

2.3. Pedagogical Trends in Automotive Electrical Training

The evolution of pedagogical approaches in automotive electrical education reflects the growing complexity and digitization of modern vehicle systems. Traditional training models, which emphasize stepwise progression from basic circuit theory to system-level integration, remain foundational. However, as illustrated in Figure 1, these models increasingly incorporate innovative instructional strategies to address gaps in practical skill development, safety awareness, and adaptive problem-solving.

One prominent trend is the integration of virtual–physical hybrid instruction, where digital simulations complement hands-on laboratory tasks. This approach enables learners to visualize complex circuit behaviors and practice diagnostics without the constraints of hardware availability or safety risks. For example, interactive circuit-building tools and real-time system fault simulators offer students the opportunity to engage in iterative experimentation, a pedagogical practice aligned with inquiry-based learning.

Flipped classrooms and project-based learning (PBL) are gaining traction in vocational education, with AI-enhanced platforms supporting real-time feedback and practical skill development in automotive electronics [3,18,25,26,27]. These methods shift instructional focus from passive knowledge reception to active, student-centered engagement. In the context of automotive electronics, PBL frameworks often involve fault diagnosis challenges, system design tasks, or wiring optimization projects, enabling learners to apply theoretical knowledge to practical, job-relevant scenarios.

AR-assisted maintenance and industry–academic collaboration platforms are being increasingly adopted to expose students to real-world tasks in controlled learning environments. By integrating wearable AR devices or remote expert support systems, trainees can receive contextualized instructions, enhancing both accuracy and efficiency. This pedagogical direction supports scalable, on-demand technical training and aligns with the digital transformation of industry practices.

Another key innovation highlighted in Figure 2 is the development of differentiated learning pathways through intelligent teaching platforms. These systems adjust instructional content based on learners’ skill progression, diagnostic accuracy, and behavioral patterns. Such platforms are especially useful in accommodating heterogeneous classroom populations with varying levels of prior experience and confidence.

Despite these advances, challenges remain. Many institutions still face resource constraints, particularly in acquiring the technological infrastructure needed for AR/VR deployment or intelligent platform integration. Moreover, assessment models often lag behind instructional innovations, necessitating the creation of capability-based evaluation frameworks that capture learners’ practical competencies, collaborative problem-solving skills, and adaptability.

In response to these challenges, the proposed AI-assisted learning platform built on Meta Llama 3 aims to reinforce the trends illustrated in Figure 1 by enabling dynamic knowledge delivery, context-sensitive diagnostics, and scalable feedback mechanisms. By doing so, it serves as a prototype for next-generation vocational training ecosystems that are adaptive, industry-aligned, and pedagogically robust.

3. Methodology

3.1. Research Framework and Design

This study is dedicated to creating an automotive electrical practice learning platform based on the Meta Llama 3 language model, with an architectural design that is highly systematic and targeted, as shown in Figure 1, Meta Llama 3 Research Framework Diagram.

The entire architecture encompasses the data layer, model layer, application layer, and interaction layer, with each layer closely related and working in concert to lay the foundation for enhancing the effectiveness of automotive electrical practice teaching. Recent studies highlight the importance of integrated architectures in AI-driven educational systems [17,18,28].

The data layer serves as the cornerstone of the platform, responsible for comprehensively collecting, meticulously organizing, and securely storing the professional knowledge system in the field of automotive electrical appliances, a variety of teaching cases, and various types of data generated by students during the learning process. This data provides solid support for subsequent model training and teaching applications.

The model layer relies on the powerful Meta Llama 3 model, and through careful fine-tuning operations, it is made to accurately adapt to the special needs of automotive electrical practice teaching. In this process, the model continuously learns and absorbs professional knowledge in the field of automotive electrical appliances, gradually enhancing its language understanding and application capabilities in this area.

The application layer integrates multiple key functional modules, among which the course content management module ensures the orderly organization and convenient presentation of teaching content; the learning diagnosis and recommendation module, with the help of advanced algorithms, provides personalized learning paths and resource recommendations based on students’ learning conditions, effectively meeting the differentiated learning needs of students.

The interaction layer focuses on building a friendly and easy-to-use interface, creating a smooth communication and efficient interactive teaching environment for teachers and students, promoting mutual growth in teaching.

3.2. Platform Implementation and Technology Stack

The AI-assisted automotive electrical practice learning platform developed in this study is implemented as a web-based application to ensure broad accessibility across various devices. The front-end is built using React.js, providing a dynamic and responsive user interface, while the back-end services are developed with Python Flask 3.11 and integrated with the fine-tuned Meta Llama 3 model through the Hugging Face Transformers framework.

The platform is deployed on cloud servers, allowing for scalable and stable operation. It supports both desktop browsers and tablet devices, facilitating flexible usage in both classroom and remote learning environments.

A screenshot of the user interface is presented in Figure 3, showcasing the main interaction panel where students engage with the AI assistant, access course modules, and receive personalized feedback.

3.3. Core Methodology for Model Fine-Tuning

To enable the Meta Llama 3 model to perform optimally in automotive electrical practice teaching, this study employs a transfer learning strategy for in-depth fine-tuning.

Firstly, a wide range of publicly available automotive electrical-related textual materials is collected, including authoritative repair manuals, professional textbooks, and cutting-edge academic papers. These vast amounts of data are used to pre-train the model, allowing it to initially construct a cognitive framework of knowledge in the field of automotive electrical appliances, becoming familiar with the professional terminology, knowledge structure, and language expression habits of the field. The model training process is as follows:

• Data Collection: Gather a large amount of automotive electrical-related texts, such as repair manuals and academic papers, to establish the model’s foundational knowledge.
• Pre-training: Conduct preliminary learning with a large-scale dataset to help the model understand specialized domain knowledge.
• Fine-tuning: Adjust the model parameters for specific teaching tasks (such as fault diagnosis, circuit analysis) to make it more targeted.
• Multi-head Attention: Enhance the model’s ability to focus on key information, improving accuracy and reasoning capabilities.

Then, based on specific teaching tasks and objectives, such as precise fault diagnosis answers and thorough explanations of circuit principles, targeted fine-tuning of the model is carried out.

In this study, we adopted the multi-head attention mechanism as a key component of the fine-tuning process to enhance the model’s ability to capture complex patterns and dependencies within domain-specific data [29].

Multi-head Attention allows the model to attend to information from different representation subspaces at different positions. Formally, each attention head computes the attention function as follows:

$$Attention(Q,K,V)=softmax({\displaystyle \frac{{QK}^T}{\sqrt{d_k}}})V$$

where $Q$, $K$, and $V$ represent the query, key, and value matrices, respectively, and $d_k$ is the dimensionality of the key vectors [29].

By employing multiple attention heads, the model can jointly attend to information from different positions and representation subspaces. In our fine-tuning process, we used 8 attention heads, which proved effective in capturing nuanced contextual cues within automotive fault diagnosis and circuit analysis texts.

This mechanism significantly improved the model’s ability to handle longer contextual dependencies and multi-step reasoning, particularly in fault diagnosis tasks involving cross-component interactions (e.g., diagnosing cascading circuit failures).

During the fine-tuning process, the multi-head attention mechanism is fully utilized. This mechanism can effectively weigh and integrate different types of knowledge, significantly enhancing the model’s ability to understand and answer complex questions. This ensures that the model can provide students with professional and accurate learning guidance, as shown in Figure 4. The model fine-tuning process is as follows:

• Task-specific Data: Select data related to teaching objectives, such as fault diagnosis or circuit principle texts.
• Model Adjustment: Update model weights to enhance task accuracy.
• Multi-head Attention Mechanism:
• Purpose: Integrate different types of knowledge to improve the model’s comprehension.
• Effect: Strengthen the model’s ability to handle complex queries. Fine-tune the model in layers to enhance domain understanding:
• Data Segmentation: Categorize data by topic, such as “fault diagnosis,” “circuit principles,” etc.
• Layer-wise Fine-tuning: Make fine adjustments based on the function of different layers to improve the model’s understanding in specific domains.

3.4. Model Fine-Tuning Process and Steps

In this study, we use the Hugging Face Transformers library to fine-tune the Meta Llama 3 model and conduct in-depth optimization for specific tasks in the automotive electrical field. The overall process includes data preprocessing, model loading, hyperparameter setting, training strategy, performance evaluation, and tuning to ensure the model has efficient and stable learning capabilities.

Data Preprocessing: Carry out strict cleaning of the collected automotive electrical text data, removing duplicate information, errors, and content unrelated to the automotive electrical field. Subsequently, perform tokenization to transform the text into token sequences that the model can easily understand. Finally, encode and pad the tokens to unify the length of all sequences to meet the model’s input requirements.

Model Loading: Conveniently load the pre-trained Meta Llama 3 model using Hugging Face’s Transformers library, preparing for subsequent fine-tuning operations.

Parameter Setting: Learning Rate: 5 × 10⁻⁵; after testing with a learning rate scheduler, 5 × 10⁻⁵ achieves a good balance between stability and convergence speed. Batch Size: 32; Training Epochs: 5; L2 Regularization: 0.01.

Model Training: Reasonably divide the preprocessed dataset into training and validation sets, and initiate model training according to the established parameters. During the training process, use the cross-entropy loss function to accurately calculate the difference between the model’s predicted results and the true labels, and efficiently update the model parameters through backpropagation algorithms, continuously optimizing model performance.

Model Evaluation: After each round of training, conduct a comprehensive evaluation of the model’s performance on the validation set, with evaluation metrics including accuracy, recall, F1 score, etc. Based on the evaluation results, adjust the parameters in time, carry out the next round of training, and continue until the model performance reaches the desired ideal state.

3.5. Verification Plan

To ensure the learning effectiveness of the model, we perform model evaluation on the validation set after each training round and dynamically adjust hyperparameters based on the results. This study employs an optimization mechanism to quickly reduce issues such as low accuracy.

If the model exhibits low accuracy, we will enhance the pre-training dataset by expanding the coverage of foundational knowledge to provide a more comprehensive learning base. In cases where the recall rate is unsatisfactory, adjustments to the model architecture will be made, specifically by increasing the weighting of error samples to help the model better capture minority classes or difficult examples. Should the F1 score remain inadequate, we plan to fine-tune hyperparameters such as the learning rate and L2 regularization strength to improve the model’s generalization ability and balance precision and recall more effectively.

4. Results

This study conducted model fine-tuning verification for an AI-assisted automotive electrical practice learning platform based on Meta Llama 3, aiming to assess its effectiveness in knowledge generation and application in the field of automotive electrical appliances. The research did not involve student testing but instead verified the model’s adaptability and stability through technical data analysis and performance testing. The following presents the research results in detail, supplemented by graphical displays of the analysis outcomes.

4.1. Knowledge Base Construction and Model Adaptability

This study successfully constructed an automotive electrical practice knowledge base, covering core concepts (e.g., circuit principles), operational procedures (e.g., fault diagnosis processes), and frequently asked questions. The knowledge base was built using approximately 500,000 words of textual data sourced from authoritative repair manuals, professional textbooks, and academic papers. The construction process involved (1) data collection from diverse, high-quality sources; (2) data cleaning to remove duplicates and irrelevant content; (3) tokenization and formatting to prepare data for model training; and (4) integration of domain-specific terminology and protocols, ensuring applicability to automotive electronics education [30,31,32]. The data sources included repair manuals, professional textbooks, and academic papers, totaling approximately 500,000 words of textual data. After cleaning, tokenization, and formatting, the data was transformed into token sequences for training with Meta Llama 3. After fine-tuning, the model can accurately recognize and generate automotive electrical jargon and knowledge points, such as “ignition coil fault diagnosis” and “working principle of relays.” In the validation set (approximately 500 question-and-answer pairs) test, the model achieved a semantic understanding accuracy of 92.3%, indicating good adaptability in the field of automotive electrical appliances.

4.2. Model Performance Evaluation

The model fine-tuning utilized the Hugging Face Transformers library with the following parameter settings: a learning rate of 5 × 10⁻⁵, a batch size of 32, 5 training epochs, and an L2 regularization of 0.01. The training and validation sets were split in a 9:1 ratio, with the validation set containing 500 automotive electrical-related questions. Performance evaluation was conducted using multiple metrics, including accuracy, recall, and F1 score, and compared with the non-fine-tuned model and Llama 2. The results are as follows.

The fine-tuned Meta Llama 3 model demonstrated notable improvements compared to its non-fine-tuned counterpart. On the validation set, it achieved an accuracy of 89.7%, representing a 12.4% increase over the original model’s 77.3%. Similarly, the recall rate improved to 87.5%, and the F1 score reached 88.6%, both reflecting a 12.5% enhancement. The integration of the multi-head attention mechanism further strengthened the model’s capacity to extract key information, as evidenced by a 15.2% accuracy improvement in tasks such as “multi-circuit voltage anomaly diagnosis.” Moreover, when compared with the benchmark model Llama 2 (also fine-tuned), the Meta Llama 3 consistently outperformed it across all metrics, achieving an accuracy of 89.7% versus 75.0%, a recall rate of 87.5% versus 73.2%, and an F1 score of 88.6% versus 74.1%, thereby demonstrating superior domain adaptability. Throughout the five training epochs, the model exhibited a steady performance improvement, with a continuous decrease in loss values, indicating effective convergence and stable learning.

The bar chart shows that the fine-tuned Meta Llama 3 significantly outperforms both the non-fine-tuned model and Llama 2 across all metrics, proving the effectiveness of the fine-tuning strategy.

The line chart shows that accuracy steadily increases with the number of training epochs, and the loss value gradually decreases, demonstrating the model’s learning capability and convergence on automotive electrical data.

4.3. Technical Stability and Limitations

The platform prototype operates stably in a single-machine environment, with an average response time of 1.8 s. However, in a simulated high-load test (processing 50 queries simultaneously), the response time increased to 6.3 s, indicating a bottleneck in server performance. Additionally, 5-fold cross-validation showed that the model’s performance is stable, with F1 scores ranging from 87.2% to 90.1%, and a standard deviation of only 1.2%, indicating good robustness.

However, the model has limitations in specific tasks:

• Mathematical Reasoning Ability: In problems involving calculations (such as “resistor value calculation”), the accuracy drops to 78.4%, which is about 15% below the expert level, possibly due to insufficient numerical content in the training data.
• Complex Problem Handling: In vague or multi-step reasoning problems (such as “Why does the circuit frequently lose power?”), the model occasionally generates incomplete answers, with the recall rate dropping to 80.1%.

The box plot shows that the distribution of F1 scores is concentrated with a small range of fluctuation and no significant outliers, proving the stability of the model’s performance.

The bar chart highlights the model’s high accuracy in semantic understanding and fault diagnosis, but its weaker mathematical calculation ability, revealing its limitations.

• Model Adaptability: The fine-tuned Meta Llama 3 performed excellently in the field of automotive electrical appliances, with significant improvements in semantic understanding and knowledge generation capabilities, meeting the adaptability standards.
• Performance Improvement: Compared with the non-fine-tuned model and Llama 2, the fine-tuned model showed a 12–15% increase in accuracy, recall, and F1 score, with a stable and convergent training process.
• Stability and Limitations: The platform operates stably under low load but requires optimization for high load performance; the model still falls short in mathematical reasoning and complex problem-solving. The results confirm the technical feasibility of Meta Llama 3 in the automotive electrical practice learning platform but also reveal room for improvement in specific scenarios. Figure 5, Figure 6, Figure 7 and Figure 8 visually display performance comparisons, training trends, and stability analysis, enhancing the authenticity and visual effects of the verification.

Task Type Evaluation and Example Cases

To clarify the evaluation methodology across different task types, the validation set of 500 question–answer pairs was organized into three subsets: 200 samples for semantic understanding, 200 samples for fault diagnosis, and 100 samples for mathematical calculation. The accuracy, recall, and F1 score reported for each task type were computed based on these corresponding subsets (

Table 2. Example questions and model responses for different task types.

Task Type	Example Question	Correct Model Response	Incorrect Model Response
Semantic Understanding	What is the function of an automotive ignition coil?	An ignition coil transforms low battery voltage into high voltage needed to create a spark in the spark plugs.	It powers the fuel pump.
Fault Diagnosis	If the car headlight flickers intermittently, what could be a likely cause?	Possible causes include a loose ground connection, corroded bulb terminals, or a faulty headlight relay.	The battery may be dead.
Mathematical Calculation	Given a 12 V battery and two resistors (R1 = 4 Ω, R2 = 6 Ω) in series, what is the voltage drop across R2?	The total resistance is 10 Ω. Current is 1.2 A. Voltage drop across R2 is 7.2 V.	The voltage drop is 6 V.

).

Representative examples of correct and incorrect responses from the fine-tuned Meta Llama 3 model are summarized below:

• Semantic Understanding Example (Correct):

Q: What is the function of an automotive ignition coil?

A: An ignition coil transforms low battery voltage into high voltage needed to create a spark in the spark plugs.

2.

Fault Diagnosis Example (Correct):

Q: If the car headlight flickers intermittently, what could be a likely cause?

A: Possible causes include a loose ground connection, corroded bulb terminals, or a faulty headlight relay.

3.

Fault Diagnosis Example (Incorrect):

A: The battery may be dead.

4.

Mathematical Calculation Example (Correct):

Q: Given a 12 V battery and two resistors (R1 = 4 Ω, R2 = 6 Ω) in series, what is the voltage drop across R2?

A: 7.2 V.

5.

Mathematical Calculation Example (Incorrect):

A: 6 V.

This additional explanation improves the transparency and reproducibility of our model evaluation results.

4.4. Comparative Benchmarking

To substantiate the technical feasibility of the fine-tuned Meta Llama 3 model, we conducted a comparative benchmarking analysis against two established baseline models: BERT-base and GPT-2. The evaluation utilized the same automotive electrical practice validation dataset employed in this study, consisting of 500 domain-specific question–answer pairs, covering semantic understanding, fault diagnosis, and mathematical calculation tasks.

The comparison was performed under identical training conditions, with each model fine-tuned for five epochs on the same preprocessed dataset using the Hugging Face Transformers framework. The primary evaluation metrics included accuracy, recall, and F1 score, ensuring consistent assessment criteria across models.

Table 3. Comparative performance of models.

Model	Accuracy (%)	Recall (%)	F1 Score (%)
Meta Llama 3 (fine-tuned)	89.7	87.5	88.6
BERT-base	81.2	79.4	80.1
GPT-2	76.5	74.8	75.6

summarizes the benchmarking results across the three models:

These results demonstrate that the Meta Llama 3 model outperforms both baseline models across all evaluated metrics. In particular, it exhibits superior domain adaptability and robustness in handling complex diagnostic queries and procedural reasoning.

To provide greater transparency and allow for verification of the reported metrics, confusion matrices were generated for the three models based on the fault diagnosis task subset (200 examples) from the validation dataset. The matrices are presented below (Figure 9, Figure 10 and Figure 11).

From the confusion matrices, we observe that the fine-tuned Meta Llama 3 model achieves a higher true positive rate and lower false negative rate compared to the baseline models, further validating its effectiveness in domain-specific diagnostic tasks.

5. Conclusions and Recommendations

5.1. Conclusions

The novelty of this study lies in its targeted application of Llama 3 fine-tuning to automotive electronics vocational training, achieving a semantic understanding accuracy of 92.3% and a 15.2% improvement in fault diagnosis tasks (Section 4.2). The proposed platform architecture, integrating data, model, application, and interaction layers, offers a replicable framework for other vocational domains [23,30,33,34]. Comparative benchmarking against BERT-base and GPT-2 (F1 score of 88.6% vs. 80.1% and 75.6%, respectively) further validates the model’s domain adaptability [28,31,32].

This study aimed to develop and verify an AI-assisted automotive electrical practice learning platform based on Meta Llama 3, focusing on the model’s knowledge generation and application capabilities in the automotive electrical field after fine-tuning. Through knowledge base construction, model performance evaluation, and technical stability testing, the study reaches the following conclusions:

• Knowledge Generation and Domain Adaptability: The fine-tuned Meta Llama 3 achieved a semantic understanding accuracy of 92.3% on the automotive electrical practice knowledge base and can effectively generate professional content, such as fault diagnosis and circuit principles. Compared with the non-fine-tuned model (accuracy of 77.3%) and Llama 2 (accuracy of 75.0%), the fine-tuned model showed an improvement of approximately 12–15% in accuracy, recall, and F1 score, proving its successful adaptation to the needs of the automotive electrical field.
• Performance Stability: The training process showed that the model’s performance steadily improved with the number of rounds, and the F1 score distribution from 5-fold cross-validation was concentrated (87.2–90.1%), with a standard deviation of only 1.2%, indicating that the fine-tuned model has good robustness and consistency. The average response time in single-machine testing was 1.8 s, meeting basic application requirements.
• Technical Limitations: The model’s accuracy in mathematical calculation tasks (such as resistor value calculation) was only 78.4%, about 15% below expert levels, indicating that its reasoning capabilities need to be strengthened. Under high-load testing (50 simultaneous queries), the response time increased to 6.3 s, revealing a stability bottleneck in the platform for multi-user scenarios.

This study did not involve student testing, but the technical verification results show that Meta Llama 3, after fine-tuning, has the potential to support automotive electrical practice learning, especially in knowledge transfer and practical guidance. However, its deficiencies in specific tasks and high-load environments also provide a clear direction for future improvements.

5.2. Recommendations

Based on the above conclusions, this study proposes the following recommendations to further enhance the performance and application value of the platform:

• Enhance Model Capabilities: To address the deficiency in mathematical computation capabilities, it is recommended to expand the training dataset to include more content related to numerical calculations (such as circuit parameter analysis) and combine it with specialized reasoning modules (such as mathematical enhancement plugins) to improve the model’s accuracy in this area. At the same time, an error self-checking mechanism can be introduced to reduce the risk of generating incomplete or incorrect responses.
• Optimize System Performance: To address stability issues under high-load scenarios, it is recommended to adopt a cloud-based distributed architecture, leveraging scalable frameworks like Kubernetes to support AI-driven educational platforms or upgrade server hardware to ensure response speed when multiple users access simultaneously [35,36,37]. In the future, larger-scale stress tests (such as 100 simultaneous queries) can be conducted to quantify the system’s capacity limits.
• Validate Practical Applications: Although this study focused on the technical level, subsequent involvement of students and teachers in automotive electrical practice courses in real testing can be introduced to evaluate the platform’s effectiveness in actual teaching. For example, by designing experimental and control groups, the impact on academic performance and practical skills can be quantified, and user feedback can be collected to optimize functional design.
• Expand Features and Content: Consider integrating virtual reality (VR) or augmented reality (AR) technology to simulate automotive electrical hands-on scenarios and enhance students’ immersive learning experiences. At the same time, regularly update the knowledge base to cover the electrical systems of new energy vehicles (such as electric cars) to keep the platform synchronized with industrial technology development.
• Explore Across Domains: In the future, the platform’s adaptability in other vocational education fields (such as mechanical engineering or electronic technology) can be tested to further expand its application scope and compare its performance with other large language models (such as GPT-4) to explore superior technical solutions.

This study has confirmed the feasibility of Meta Llama 3 in an automotive electrical practice learning platform through technical verification, laying the foundation for the application of artificial intelligence in vocational education. With the continuous optimization of the model and system, the platform is expected to become an innovative tool for improving teaching efficiency and students’ practical capabilities.