Understanding the Impact of a Learning Management System Using a Novel Modified DeLone and McLean Model

Abstract

Due to the evident importance of e-learning in education, the number of e-courses and systems delivering various services has exploded. Thus, evaluating e-learning systems is critical to ensuring their successful delivery, practical use, and beneficial effects on learners. Therefore, this study aims to evaluate the success of learning management systems using the modified DeLone and McLean model. A survey involving 239 in-service vocational teachers evaluated the research model. Statistical analysis was used to examine the hypotheses, specifically linear regression analysis. The results of the nine proposed hypotheses indicate that two had an insignificant relationship: information quality on user satisfaction (H5) and system quality on use (H7). The finding shows that the proposed model may give an alternative point of view to the LMS evaluation.

1. Introduction

A learning management system (LMS) has recently become popular, especially for learning activities [1]. LMS is a software application for general educational purposes such as administration, documentation, reporting, automation, learning, and human resource development program [2,3]. The LMS concept is developed for e-learning and has many benefits in its implementation.

Based on the description, one LMS application is for the human resource development program. One standard human development program is teacher training and development in this context. This program is designed for the pre-service or in-service teacher to develop pedagogic and technical skills. On the other hand, many countries have implemented programs for teacher development. In Nigeria, the program provides a sustainable program to renew knowledge, skill, and competency. It can be achieved by conducting on-the-job training, such as induction programs, ICT training, workshops, conferences, and seminars, particularly for non-professional teachers employed in schools [4].

In Serbia and Russia, the program is completed by implementing digital competency to develop vocational teacher professional digital competency. Through the program, teachers are qualified to teach in a local, national, or international work environment following the European Union’s policy on online learning [5]. A specific model of LMS for Serbia and Russian teacher are difficult to find. Therefore, there should be initiative among the educated in terms of developing the required LMS.

In the Philippines, the program was realized through practicum teaching experience. In Philippine education, all teacher education curricula at all higher education institutions include practical teaching as one of the requirements for obtaining a Bachelor of Education degree [6]. In their research, they found that most of the issues and challenges were connected to what can be described as the pre-service teacher training program in the Philippines, as reflected in the practicum experience of the pre-service student teachers.

In Argentina, a teacher training program is conducted to use digital technology for mathematics teachers. It was completed to refine the needs of future teachers and the role of technology in advancing mathematics embodied in project modeling [7]. However, students may need to integrate LMS with real-world experience, for example, in a remote laboratory [8]. The integrated infrastructure may revalue student and teacher communication in face-to-face teaching and avoid capitalizing on the digital competencies of teaching hybridization.

In Australia, the teacher training program is carried out by teaching and learning in the Australian context during practicum and how this learning experience constructs their identity as teachers through the activity theory framework [9]. In addition, using technology to transform online learning and teaching with the technological pedagogical content knowledge (TPACK) framework [10]. However, a few common problems can arise when using an LMS in Australia. Some key issues and potential solutions are technical, lack of training, inadequate content, limited collaboration, and accessibility [11]. Overall, the successful implementation of an LMS requires a collaborative effort between institutions, faculty, and students. By addressing these common problems, institutions can create a more effective and efficient learning environment for their students.

In Indonesia, the teacher training program is well known as the Pendidikan Profesi Guru (PPG) program. This program is a continuation for undergraduate or applied undergraduate students, demanding different graduate competency standards. The competency standards for graduates of the PPG program include attitudes, knowledge, and skills. The PPG program is held together with 61 universities in Indonesia in its implementation. The total quota of training participants was 24,000 teachers in 2020 and 23,961 teachers in 2021 [12]. Based on the significant involvement of teacher professional education training and the broad geographical reach throughout Indonesia, the benefits of using this system management learning program need to be evaluated. Aspects of differences in devices, internet connection, background knowledge, and education require a study that looks at the contribution of each aspect. The purpose of holding PPG is to prepare more competent teachers in their fields. Therefore, there are several advantages to participating in the national program, namely (1) having a certification of expertise according to a nationally recognized field; (2) having a greater chance of becoming a civil servant teacher; (3) obtaining professional certification allowance from the government; (4) increase relations with teachers and lecturers throughout Indonesia.

The LMS is globally used but the implementation is unique based on its user characteristic. Facing the 21st century, UNESCO, through the journal “The International Commission on Education for the Twenty-First Century,” recommends continuous-lifelong education based on the four pillars of the learning process: learning to know, learning to do, learning to be, and learning together [13]. To realize the four pillars of education in the current era of information globalization, teachers as learning agents need to master and apply Information and Communication Technology in learning. This statement from UNESCO became important during the pandemic to continue developing a learning management system, especially in the Indonesian government program, namely PPG. As technology continues to advance and the importance of digital learning grows, there are several emerging trends in the impact of LMS: personalized learning [14], gamification [15], mobile learning [16], social learning [17], analytics [18], and data-driven insights [19]. Overall, the emerging trends in LMS design focus on creating a more engaging and personalized learning experience that is accessible, collaborative, and data-driven, which is considered as convoluted problems.

Despite the benefits of implementing the LMS, it has some common problems. One of them is the high level of student absenteeism compared to face-to-face learning. The assumption of this problem is the level of user acceptance of technology, including the user’s willingness to use technology. The differentiating factor, in this case, is the level of knowledge and technology application of different users. In addition, there are research findings that reveal that online learning program management can be realized effectively if the teacher carries out his duties professionally so that students get a sense of satisfaction and are loyal to the learning activity itself, which in the end, the goals of the teaching-learning process can be fulfilled [20]. Therefore, it is essential to know the aspects that influence the acceptance of LMS users to improve the student learning experience in the Indonesia Teacher Training Program (PPG). Some of the existing related literature found several limitations from the previous studies: lack of personalization, limited interaction, technical issues, cost, and limited flexibility. Therefore, this study aims to promote the shortest path yet novel modification DeLone and McLean (D&M). The model is used to discover the user’s understanding of LMS’s impact. Our proposed model sees LMS users and developers as equal entities.

This study raises the main question of whether the proposed model can indicate the user experience. This study is divided into several chapters to understand the content better. The theoretical framework explains the LMS theory, the default of the DeLone and McLean model and each modification, and the research hypotheses (H1–H9). The next chapter explains the research method, which consists of data collection, analysis, and interpretation. We present the result in a statistical report for each hypothesis. In the discussion, we promote Net-Benefit and its influential factors as a consequence of the modified DeLone and McLean model. The final chapter concluded the result and recommended it for future research.

2. Theoretical Framework

Learning Management System (LMS)

LMS is a web-based system that supports educational purposes in general and learning in particular. This system combines several media, such as audio, video, text, and graphics. The primary purpose of the LMS is to complement face-to-face learning and allow the development of more flexible learning methods. LMS is a platform that uses the internet as a data transfer to process, store and disseminate learning resources. Additionally, it supports administration and communication related to education and learning. This approach allows students and teachers to adjust the learning intensity suitable to their needs. Figure 1 presents the main page of PPG’s LMS.

This LMS application includes several essential features that facilitate online learning. The course is the primary menu item students access during the learning process. There are various sub-menus of the LMS depicted in the Figure 1. The system will present an overview of the lesson, schedule, table of contents, and learning materials in the content sub-menu. Announcements from lecturers/instructors will be displayed in the announcements sub-menu. The MyBrightspace sub-menu includes shortcuts to several of Brightspace’s features. Participants can view their learning progress in this menu by selecting the Class Progress option. Additionally, the topics sub-menu is utilized to access the class’s discussion forums. Participants can quickly communicate with teachers and classmates using this option.

Additionally, the class list sub-menu will provide a list of class attendees. The instructor will be able to communicate with participants rapidly via email or instant chat via this option. The course tools sub-menu is a somewhat sophisticated menu, as it has various facilities that participants can access, from seeing assignments and grades to participating in the Virtual Classroom via video conference. The final sub-menu is the FAQ (Frequently Asked Questions). The user can access this menu to view which topics are often asked questions on the LMS and the answers to these queries.

3. DeLone and McLean Model

LMS, as part of an information system, can be measured in the same way as the success and quality of IS. One model that can be used to measure it is the updated DeLone and McLean (D&M) model [21]. This model provides a comprehensive understanding of IS success by identifying, describing, and explaining relationships among the six main dimensions in IS that are generally evaluated. The six main dimensions are system quality, information quality, service quality, use, user satisfaction, and net benefits. Figure 2 shows the model and relationships between dimensions from the updated D&M.

4. Proposed Model and Hypotheses

This study uses the updated D&M model base to evaluate the LMS. This model will be used and modified according to the needs and characteristics of the LMS. As the basis of the model, the net benefit is the main component in measuring IS success. It is related to cost savings, expanded markets, incremental sales, reduced search costs, and time savings. However, since this study does not measure the profit-oriented LMS, it will measure the benefits from the user’s perspective. Therefore, modifications of the initial D&M model were carried out to suit the LMS evaluation. Figure 3 is a model and the relationship between constructs that have been modified.

From Figure 3, the net benefit is not the final construct or the key to the success of an IS. This construct equals use and user satisfaction as an indicator of IS success. Meanwhile, system quality, information quality, and service quality remain the constructs used to measure IS success. The modification of the D&M model illustrates that system benefit or advantage variables will affect the impact variables, results, and benefits provided by the system on user needs and the success of the PPG LMS. From the research model, several hypotheses are created and subsequently examined:

Hypothesis 1 (H1).

System Quality Is Positively Associated with the Use of LMS.

Hypothesis 2 (H2).

Information Quality Is Positively Associated with the Use of LMS.

Hypothesis 3 (H3).

Service Quality Is Positively Associated with the Use of LMS.

Hypothesis 4 (H4).

System Quality Is Positively Associated with User Satisfaction in LMS.

Hypothesis 5 (H5).

Information Quality Is Positively Associated with User Satisfaction in LMS.

Hypothesis 6 (H6).

Service Quality Is Positively Associated with User Satisfaction in LMS.

Hypothesis 7 (H7).

System Quality Is Positively Associated with a Net Benefit in LMS.

Hypothesis 8 (H8).

Information Quality Is Positively Associated with a Net Benefit in LMS.

Hypothesis 9 (H9).

Service Quality Is Positively Associated with a Net Benefit in LMS.

5. Method

Quantitative approaches based on the modified D&M model are employed to examine theoretical models and hypotheses, and this study used a quantitative analytical survey. In general, Figure 4 presents the research flow in this study. The study generates nine hypotheses with the correlation of each construct in the modified D&M model. Figure 4 presents the research flow in this study.

Data collection is essential to this research flow, providing the information necessary to answer research questions and test hypotheses. A general overview of the data collection process in research is developed data instrument, for example, as in

Table 1. Questionnaire for PPG LMS user.

No.	Variable	Item
1	System Quality	I find the PPG LMS easy to use
2		I find it easy to get the PPG LMS to do what I want
3		The PPG LMS is flexible to interact with
4		Learning to operate the PPG LMS was easy for me
5		I find the PPG LMS website design is interesting
6		It is easy for me to do the exercise using technology (PPG LMS)
7		User privacy is provided in PPG LMS
8		PPG LMS gives notification if there something happens with the system.
9	Information Quality	The information generated by the PPG LMS is correct
10		The information generated by the PPG LMS is useful for its purpose
11		The PPG LMS generates information in a timely manner
12		I trust the information output of the PPG LMS
13		The information in PPG LMS provides me with the opportunity to practice
14		Easy to find the information that I need
15	Service Quality	There is adequate technical support from the system’s provider
16		The overall infrastructure in place is adequate to support the PPG LMS
17		The PPG LMS can be relied on to provide information when needed
18		The output of the PPG LMS is complete for study processes
19	Use	Using the PPG LMS enables me to accomplish tasks more quickly
20		Using the PPG LMS has improved my study performance
21		Using the PPG LMS has made my study easier
22		I like doing exercises in PPG LMS
23		I think exercises done on PPG LMS are beneficial
24		I find the PPG LMS useful in my study
25		Easy to submit the tasks
26	User Satisfaction	I am satisfied with the functions of the PPG LMS
27		I am satisfied with the learning material provided in PPG LMS
28		The video quality is excellent
29		The PPG LMS has eased work processes
30		I am generally satisfied using the PPG LMS
31	Net Benefits	The PPG LMS will help overcome the limitations of the paper-based system
32		Using the PPG LMS will cause an improvement in my teaching performance
33		The PPG LMS facilitates easy access to teaching materials
34		The PPG LMS will enhance communication among users
35		PPG LMS use will cause improved decision making
36		PPG LMS help me complete my task effectively

. In the table, we used 36 questions for data collection. The instrument is used to collect data from participants, and it can take many different forms depending on the research question and the type of data being collected.

The next step is data analysis. Data analysis aims to identify patterns, relationships, and trends within the collected data and to use these findings to answer the research question or test the research hypothesis. In this study, the analysis uses a t-test and R Square. The primary function of a t-test is to determine whether there is a significant difference between two groups. R-Square is used to determine the usefulness of a regression model.

The last process is data interpretation. Data interpretation is used to analyze and make sense of the data collected during the research process. Data interpretation aims to draw conclusions and make inferences from the data that can be used to answer the research question or test the research hypothesis. In this study, we proposed a modified DeLone and McLean model to interpret the data.

The measuring instruments were chosen based on six primary updated D&M constructs. The instrument will be evaluated, arranged into a questionnaire, and given to the respondent. The research population is students of PPG academic years 2020 and 2021 who use LMS as a learning platform. Sampling was conducted using a snowball sampling technique due to the researcher’s limitations regarding the geographical distribution of respondents who passed the PPG program. The PPG training participants are distributed over 27 provinces, resulting in a sample size of 239 in-service teachers who take the PPG program. Of the existing participants, 124 were female and 115 male, aged around 25–32 years.

The survey was conducted with students enrolled in the LMS supplied by the Indonesian Ministry of Education and Culture. LMS is used to support teaching and learning materials and activities and provides a variety of interactive activities such as forums, wikis, quizzes, surveys, chat, and peer-to-peer activities. Through a questionnaire, respondents were asked to rate the criticality of each aspect of the model on a five-point scale. Additionally, an open-ended question was included to allow respondents to express their thoughts on variables not covered by the closed-ended question.

Before sending the questionnaire to respondents, the validity, reliability, and analytic capabilities were evaluated. The questionnaire’s validity was evaluated using 30 respondents to determine the questionnaire’s correctness or efficacy. The validity test revealed 36 items that satisfied the criterion for representing six variables in the modified D&M model. The D&M modification model can cut the path from system quality, information quality, and service quality dimensions to net benefits more quickly when compared to the original model, which must first pass use and user satisfaction. The modified D&M can be used to answer a variety of questions regarding the efficacy and productivity of the PPG LMS. The evaluation may incorporate metrics such as system dependability, information correctness, reaction time, and user feedback to answer the question. The model can be used to analyze the system’s net benefits. This review may incorporate cost savings, time savings, and increased efficiency to address the question. Nevertheless, the M&C model may be used to evaluate the success of a PPG LMS and provide answers to numerous concerns concerning its efficacy and efficiency. The metrics and methodologies used to evaluate the quality criteria can be adjusted to the precise topic posed. The research plan, including the questionnaire, has been presented in front of the research reviewer at Universitas Negeri Malang. Even though it is not following with the issue of ethical clearance, the acceptance of the university reviewer means that this study follows ethical conduct to avoid harm and research misconduct [22].

The primary and classical assumption tests were used to determine the significance of each variable. Then the multiple linear regression analysis stages were performed as a hypothesis test, namely the t-test and coefficient of determination (R²). This analysis aims to ascertain the LMS’s quality and success rate for PPG within 2020–2021. Furthermore, the study generates nine hypotheses described in the previous section. The hypotheses’ results will be revealed and discussed in the final stage.

6. Result Analysis

The t-test determines the correlation between the independent and dependent variables in the D&M model, with a decision-making t-count larger than 1.970 and a significance value of 0.05 or less. The results of the t-test for each of the nine proposed hypotheses are shown in

Table 2. t-test result.

	Y1	Y2	Y3
X1	0.000	0.009	0.000
X2	0.039	0.123	0.005
X3	0.107	0.001	0.000

. According to Table 1, there are two rejected hypotheses which are H5 and H7.

The next test is to determine the determination coefficient (R²). This test quantifies the amount of the influence of predictor factors (X1, X2, and X3) on the variables of use (Y1), user satisfaction (Y2), and benefits (Y3) simultaneously. The conclusion between the models is shown in

Table 3. R² Result.

Model	R2 Value
1 (X1 + X2 + X3 → Y1)	0.211
2 (X1 + X2 + X3 → Y2)	0.135
3 (X1 + X2 + X3 → Y3)	0.631

. As shown in Table 2, model 1 has a simultaneous effect on the use (Y1) of 21.1%, model 2 has an effect on user satisfaction (Y2) of 13.5%, and model 3 has an effect on benefits (Y3) of 63.1%.

Hypothesis 1 (H1).

System quality is positively associated with the use of LMS.

The hypothesis test results imply a positive correlation between system quality and use. System quality is defined as an SI’s reliability, convenience, ease of use, and functionality. This factor influences utilization in terms of usage patterns and intensity of use. Although this hypothesis has a considerable influence, if the system quality variable is increased, it will directly affect the usage variable or the behavior of student instructors when using the LMS. The test results corroborate research indicating that the system quality and use variables are inextricably linked to the effectiveness of an information system [23,24,25].

System quality is a critical component of ensuring the LMS’s sustainability. Users are extremely concerned with the efficiency of a system that enables them to generate high-quality online activities by using a quality system. Regardless of the situation, the user requires a high level of system quality. This means that information technology suppliers must emphasize the relevance of quality components, particularly quality systems. Lack of quality has been a significant oversight of information technology suppliers. This failure has resulted in system users refusing to use the new system, resulting in disadvantages for shareholders.

Hypothesis 2 (H2).

Information quality is positively associated with the use of LMS.

The second hypothesis is accepted, implying that information quality influences LMS usage. A large number of users in Indonesia necessitates accurate and perfect information regarding LMS use during training. Completeness, readability, personalization, relevance, and security are all characteristics of high-quality information. The test results demonstrate that high-quality information has a considerable impact on how system users (use) information to accomplish their goals [26,27].

The primary role of any information system is to provide valid information to its users. The quality of the information provided by the system could be assessed by semantic success measures such as timeliness, accuracy, completeness, consistency, and relevance [21]. Providing a higher level of information quality will produce a higher level of satisfaction with the system. Information quality was among the core factors influencing user satisfaction for instructors and students [28,29]. Furthermore, Information quality has been found to have a significant positive influence on students’ satisfaction with using LMS [3,30,31,32].

Hypothesis 3 (H3).

Service quality is positively associated with the use of LMS.

The level of services offered by LMS administrators to PPG students affects students’ perceptions of the usefulness of the learning system and their usage of it. The quality of a service is directly tied to the administrator’s responsiveness to the user. The more responsive the administrator of a system is, the greater the influence and usage. Clear instructions and online assistance are hallmarks of a high-quality service. Additionally, the administrator is always prepared and cooperative when system faults occur. The test results corroborate current findings that a substantial relationship exists between an information system’s service quality and use variables [24].

Hypothesis 4 (H4).

System quality is positively associated with user satisfaction in LMS.

The hypothesis’s findings indicate a positive correlation between the influence of the system quality variable and user happiness. Aspects of user satisfaction that are related to the technical quality of the system include the ease with which the learning system can be used, the system’s ability to meet the needs of users, the system’s ability to interact, the integration and consistency of the system’s various components, and the presence of features and functions required by the users. A high-quality online learning system can match students’ expectations for LMS use in this scenario. These findings validate previous studies that found a substantial effect on system quality and user satisfaction [29].

This finding demonstrates that people value factors, such as interaction, navigation speed, and user interface, when determining their level of satisfaction. When users perceive that the LMS is engaging, has an intuitive layout that makes information easily accessible, and responds swiftly to any issues, they are more satisfied with their experience. Similarly, the system’s availability and the supporting team should reply to students’ inquiries and offer them the assistance they seek, increasing their level of satisfaction.

Hypothesis 5 (H5).

Information quality is positively associated with user satisfaction in LMS.

The primary function of the LMS is to give students pertinent, accurate, and up-to-date information about their academic materials. If the LMS meets the students’ demands by ensuring they have access to information when needed, they will be more satisfied with the system. Additionally, it demonstrates that students understand the value of the material offered through the LMS and are content with the present level of quality available through the LMS [33].

However, the hypothesis’s results indicate that the quality of information does not significantly affect user happiness. It suggests that PPG training instructors who are competent in Information and Communication Engineering already understand a great deal about competency in general and hence have a slight direct impact on user happiness. The test results corroborate study findings that information quality and user pleasure have no discernible influence [34].

Hypothesis 6 (H6).

Service quality is positively associated with user satisfaction in LMS.

The study’s findings indicate a positive correlation between service quality and user satisfaction. These findings suggest that providing students with high-quality services can raise their satisfaction with the LMS. Thus, it is critical to have technical personnel on hand when needed, to maintain control over the technology, to assist students by offering guidance and training on how to use the system, and to be able to resolve technical issues students have with the LMS. This will meet their demands, promote pleasant emotions, and improve their happiness with the system. The test results are consistent with prior research, which indicates that the relationship between service quality and user satisfaction is essential and helps achieve learning objectives [35].

Service quality refers to the assistance that an information system’s service provider may deliver [36]. It also refers to the timely availability of various communication channels to assist users in resolving LMS-related issues. The provision of system support services to users is seen as one of the most critical tasks, as system support significantly impacts the system’s successful use.

Hypothesis 7 (H7).

System quality is positively associated with a net benefit in LMS.

System quality refers to the capacity of an IS to provide technical support for online activities in an LMS efficiently. The system quality measure evaluates the website’s performance; some metrics are page load speed, ease of navigation between pages, website design, appearance, website availability, and website layout. The performance of an LMS is critical in determining the user’s gain from the system. In general, system quality has a beneficial effect on the net advantages of LMS.

However, the quality of a system is not always proportionate to its benefits. A system of low quality may yield excellent benefits and vice versa. The system’s quality can be determined in various ways, including adaptability, availability, reliability, reaction speed, and usability [23,37]. Teachers are considered to have experience using information systems in the case of this LMS. Thus, the user’s adaptability is already quite good. As a result, users can still profit even if the system has shortcomings in terms of availability, dependability, response time, and usability.

Hypothesis 8 (H8).

Information quality is positively associated with a net benefit in LMS.

The hypothesis establishes a positive association between the quality of information and its advantages. The quality of information is a factor in determining the utility of an information system. For instance, providing appropriate, precise information, frequently updated content, and beautiful content designs is critical to ensure students have a positive experience. Additionally, organizing content and information into e-learning systems enables students to perform learning activities more efficiently. The test results show that the information quality variable significantly affects the net benefit variable [38].

Hypothesis 9 (H9).

Service quality is positively associated with a net benefit in LMS.

The findings of the tests indicate that there is a good correlation between service quality and benefits. According to data collected from participants, the service quality variable will directly affect the benefit variable. The favorable impact teachers working in the field of ICT have on their usage of the national LMS, given that service management is implemented in a stratified sectoral fashion. The test results support previous research, indicating that the service quality variable significantly affects the benefit variable.

7. Discussion

This paper evaluates the promote the shortest path yet novel modification DeLone and McLean (D&M). The model is used to determine the user’s comprehension of the LMS’s effects. Based on the statistical result and interview, the proposed method directly contributes to System Quality, Information Quality, and Service Quality to a net benefit in PPG’s LMS. Figure 5 shows and concludes the findings.

The existence of a quality control system determines the system’s quality. An effective quality control system lowers production costs by reducing material costs and requiring less rework. Quality control activity may be a double-edged sword for system developers. Developers understand the system’s standards well, try to improve them, and produce quality goods to their best. Thus, it improves the morale of the developers. On the other hand, too light or tight control may reduce the system quality, which aligns with developers’ morals and user satisfaction.

High-quality information should be objective and free of bias. There should be no mathematical or grammatical faults. However, information accuracy is not sufficient. It should also be comprehensive, implying that no facts or numbers should be omitted or hidden. Telling the truth but not entirely is pointless.

The advantages of information should be weighed against the expense of getting it. Spending money on information that cannot be recovered is not worthwhile in a commercial environment, resulting in loss each time received information. However, gathering information might be beneficial in educational situations even if there are no financial incentives due to the learning environment.

On the other hand, information should be delivered with structure, depth, and complexity that meets the information demands of the users. For example, the Head of the School needs succinct reports to grasp the position and performance of instructors at a glance, whereas program directors want thorough information to make day-to-day actions.

The information must be transmitted to the appropriate person. In this case, the LMS should govern the decisions expected to collect the information. As a result, information should come from the system as trustworthy sources. It is determined by the LMS’s credentials, experience, and track record of providing information.

Timely communication of information is essential. What constitutes timely information varies depending on the scenario. Choosing an appropriate communication channel is a critical skill to master. Due to the geographical condition of Indonesia, where internet access is not widely spread, this function should be enhanced in the future.

A good LMS should give its users exceptional services. Naturally, when teachers, as the prominent users, are satisfied with the LMS service, they will continue with it and not look for alternatives. This is a significant advantage for the provider because user retention is considerably less expensive than user acquisition.

A good level of service motivates teachers to spread the word about the LMS. After obtaining high-quality service for a while, they are more inclined to do so. These recommendations are helpful to the developer. The more referrals collected, the more users we will gain without paying any money. Although word of mouth may reach many individuals, not everyone will be the user.

Nonetheless, excellent word of mouth may be generated by providing outstanding service. When a user compliments the LMS, it enhances the morale of the creator and supplier. Naturally, confidence grows with time, which may enable the service to keep instructors satisfied.

These three factors directly contribute to the net benefit, which is not captured in the previous studies. This novel modification implies future strategies for LMS development and services. The strategies could be different if the developer implements the original DeLone and McLane Models. The direct contribution perceives the net benefit as equal to user satisfaction and intention to use. In other words, the model can capture the LMS condition modified based on the equality of all entities: providers and users. The modified D&M model also can solve the limitation of LMS in previous work, such as lack of personalization, limited interaction, technical issues, cost, and limited flexibility. This proposed model could be implemented in future research related to the user experience of LMS or other content management systems.

8. Conclusions

Using a modified D&M model, this study demonstrates significant results in LMS applications. Additionally, it equips higher education institutions with credible, trustworthy, and comprehensive models and procedures for assessing the LMS’s success. It is feasible to design applications with similar qualities based on the study’s findings regarding the link between factors. Additionally, this study can assist educational institutions in identifying system characteristics such as ease of use, system reliability, personalization, and integration between system components that should be improved to make the system more reliable, user-friendly, personal, attractive, intuitive, and simple to navigate. These factors should contribute to the system’s perceived usefulness and enjoyment. A periodic assessment using this modified model should be conducted to evaluate the fitness of LMS with the user needs. Once it is unsuitable a development based on the gaps should be performed.

However, the modified D&M only focuses on the technological aspect, these research cannot provide the non-technological aspect, such as user behavior and external factors. The model developed in this work can serve as a springboard with additional non-technological aspects for future research. Researchers can explain, evaluate, and compare differences in outcomes. Given the rapid evolution of technology and e-learning, long-term and ongoing research into how the e-learning quality characteristics identified in this study change over time may offer valuable information. Finally, the modified model can be used as a new general metric for LMS evaluation.

This research viewpoint may be slightly more applicable to STEM fields, but it is not applicable to many other fields, such as those that place a more nuanced emphasis on the interpretation and production of information within the context of social settings. Hence, future research into how LMS may be used to enhance collaborative learning would be of greater practical benefit to educators than the measurement of components surrounding a theoretical model would be of help to them.