Next Article in Journal
Ecocentric Education: Introduction to a Special Collection of Essays
Next Article in Special Issue
Exploring College Student’s Perspectives on Global Mobility during the COVID-19 Pandemic Recovery
Previous Article in Journal
Measurement of the MOOC Phenomenon by Pre-Service Teachers: A Descriptive Case Study
Previous Article in Special Issue
Tutorials for Integrating 3D Printing in Engineering Curricula
Article

E-Learning Critical Success Factors during the COVID-19 Pandemic: A Comprehensive Analysis of E-Learning Managerial Perspectives

Department of Industrial Engineering, King Abdulaziz University, P.O. Box 80204, Jeddah 21589, Saudi Arabia
*
Author to whom correspondence should be addressed.
Educ. Sci. 2020, 10(9), 216; https://doi.org/10.3390/educsci10090216
Received: 19 July 2020 / Revised: 18 August 2020 / Accepted: 19 August 2020 / Published: 20 August 2020

Abstract

During the COVID-19 pandemic, educational institutions were shut down all over the world, which impacted over 60% of students and caused a massive disruption of the education system. The goal of this paper was to identify the critical success factors for E-learning during COVID-19 using the multi-criteria Analytic Hierarchy Process (AHP) and Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) techniques to enhance the educational process. Data were generated by interviewing 69 E-learning managers in educational institutions during COVID-19 based on defined evaluation criteria and E-learning approaches through several channels. We found that technology management, support from management, increased student awareness to use E-learning systems, and demanding a high level of information technology from instructors, students, and universities were the most influential factors for E-learning during COVID-19. Among the five learning systems, blended learning was the most suitable learning system to practice. These results demonstrated that, regardless of how extraordinary the technology is in an educational institution, the readiness of E-learning execution played a large role in boosting the educational process during the COVID-19 pandemic.
Keywords: e-learning; critical success factors; distance learning; COVID-19/Coronavirus pandemic; AHP-TOPSIS e-learning; critical success factors; distance learning; COVID-19/Coronavirus pandemic; AHP-TOPSIS

1. Introduction

Prior to the COVID-19 pandemic, E-learning was growing approximately 15.4% yearly in educational institutions around the world without uncertainties or pressure on those institutions or on students [1]. However, as this research was conducted during COVID-19, the situation has changed dramatically. Educational institutions began providing most of their services online, including lecturers and different assessments via several platforms for over 60% of students around the world due to global restriction measures to minimize the spread of COVID-19 [2]. Referring to the data released by the World Health Organization (WHO), COVID-19 has been reported in over 216 countries, and there are areas with millions of confirmed cases [3]. Many countries have taken precautionary measures, including lockdowns of schools and universities, and switching to full E-learning mode during the spread of the Coronavirus, to avoid future expected waves [4]. This action was in response to social distance rules, which were strongly recommended by the WHO to prevent the spread of COVID-19 [5]. This lockdown began in the middle of the spring semester, which was unplanned for both instructors and students.
However, many studies have previously examined the critical success factors (CSFs) in the education sector from both the instructor’s and students’ perspectives for future improvement in the E-learning system. Organizations can determine the most valuable CSFs that should be achieved in order to boost a project mission. Thus far, these studies examined the CSFs of E-learning during typical times. Yet, the CSFs during the COVID-19 pandemic are expected to be different than the CSFs during typical times for many reasons. Firstly, during COVID-19, the switch to E-learning was for all educational institutions, which was unplanned. Not all institutions had the ability to switch smoothly as not all of them were previously implementing E-learning, unlike institutions that already offered E-leaning and were planning for and investing in the E-learning process. Secondly, during COVID-19, many factors other than educational ones, such as political and health factors, influence the process, which makes it an abnormal situation. For example, during typical times, students may visit the library, attend tutoring sessions, and even go to places with a good internet connection speed if they do not have a good internet connection at home, unlike during COVID-19 where students were in curfew situations. Thirdly, the course material of the classes that were taught through E-learning pre-COVID-19 were well-prepared, unlike during COVID-19 where courses were not planned to be taught through E-leaning.
The list could continue, not only for the CSFs but also for E-learning system approaches. These are just some of the main differences worth evaluating during the pandemic. In this paper, we discuss the essential success factors from the perspective of E-learning managers in various educational institutions using multi-criteria decision-making methods to ensure the continuity of educational objectives and students’ prosperity in their education while fulfilling the World Health Organization (WHO) social distancing recommendations. This provides the perspective of E-learning managers, which allows us to understand the best practices during uncertain crises that could force educational institutions to switch to E-learning. This can help policy makers in educational institutions to better execute the educational process during a crisis through improving the most critical factors to prepare for.

2. Literature Review

2.1. E-learning

E-education, distance-learning, and online learning are all different terminologies of E-learning. The authors in [6] defined E-learning as “the wide set of applications and processes which use available electronic media and tools to deliver vocational education and training”. Researchers [7] stated that E-learning is "the use of various technological tools that are web-based, web distributed, or web capable for education". E-learning has been growing year after year as there are many advantages, such as flexibility, internet accessibility, and cost-effectiveness [8]. These advantages could transform education into a lifelong learning process. According to [9], having access to lectures anytime, as many times as needed, allows students to better recall the information that is required for traditional education.
The flexibility of E-learning is a solution for people’s commitments to their family or work, which may increase the number of people who enroll in this type of education. In fact, this goes beyond the learners; it gives flexibility also for the instructors. In addition, educational institutions are implementing E-learning technologies to improve the communication among learners and instructors for better knowledge exchange as well as to strengthen the learning community to accomplish personal objectives [10].
In 2005, the Kingdom of Saudi Arabia established the National Center of E-learning and Distance Learning (NCEDL), in which at least nine universities were involved [11]. This central role was aimed at enhancing the experience of E-learning in educational institutions by adapting and implementing the most effective practices of the E-learning system globally [12]. According to the National Center of E-learning and Distance Learning, the NCEDL has been involved in several E-learning system projects, such as the Learning Portal, which helps students to access the online learning material remotely as well as train teachers in the use of E-learning tools. Furthermore, the center has created an Award for Excellence of E-learning to encourage educational institutions to utilize E-learning, which involves around 42 institutions.
In 2011, the government of Saudi Arabia established the Saudi Electronic University (SEU), with tens of thousands of students enrolled in its different programs, including undergraduate and graduate studies. Since the occurrence of these events, King Abdulaziz University implemented many technological tools in order to enhance their practice in the E-learning system, such as the Learning Management System (LMS), which supports fresh and junior students by providing access to over 16,000 e-books as well as other academic resources online [12]. All these efforts for E-learning impacted the education transformation from on-campus to distance learning during the COVID-19 pandemic, which happened suddenly without enough time to plan.

2.2. TOPSIS and E-learning

In 1981, Hwang and Yoon developed the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) method [13]. This method is commonly known for the Multiple Criteria Decision-Making practice to determine the best alternative among a set of alternatives [14]. This method essentially determines the distance of both the positive and the negative alternatives of the ideal solution [15]. The TOPSIS technique is widely used in many industries, including education; specifically, E-learning is becoming more popular and is growing over time.
For instance, researchers in [16] evaluated the criteria of five different approaches of E-learning, where each approach’s performance was rated and computed using the TOPSIS method. They concluded that the Flipped Classroom, a student-centered approach with online material provided to student’s prior to classes, was suggested as the most convenient E-learning approach, whereas ‘strategic readiness for E-learning implementation’ was ranked as essential criteria. Additionally, in [17], the authors evaluated and selected the learning objectives of e-content and educational material in web-based learning systems. They combined both the Analytic Hierarchy Process (AHP) and TOPSIS methods to ease the selection process. They suggested that the learning outcomes metadata is an effective procedure to produce e-content.
Eight criteria were evaluated using the Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE) and the TOPSIS method to examine the E-learning readiness as well as to weight the criteria based on Simple Additive Weighting (SAW) and AHP to determine the weaknesses and improve the implementation of E-learning [18]. Indeed, TOPSIS has been commonly used to evaluate different criteria and factors in E-learning as well as different educational system approaches. Therefore, conducting this technique will help to find the most CSFs for E-learning during COVID-19.

2.3. Analytic Hierarchy Process (AHP) and E-learning

In 1980, Saaty [19] developed the Analytic Hierarchy Process (AHP). This is an effective decision-making process where quantitative and qualitative features are considered [16]. When the decision relies on several criteria, the AHP method is one of the most effective techniques to use [16]. Although the method has been criticized over its process, almost all fields, including resource allocation, management, and education, have used this technique for making important and responsible strategic decisions [20]. For example, Dweiri [20] researchers investigated and prioritized the critical factors of violations of academic integrity in Saudi Arabia within educational institutions. They stated that twelve essential factors of E-learning were prioritized using an analytic hierarchy process. They concluded that the most critical factor was the inappropriate guidelines provided to students, whereas a shortage of feedback was the least critical factor amongst all factors.
In [21], the authors evaluated the CSFs of E-learning from both the instructor and student perspectives at Sebelas Maret University. They stated that they used AHP and fuzzy techniques to determine the ranking of the CSFs from both perspectives. They concluded that the five CSFs from the instructor’s point of view were fiscal policy, regulatory policy, course quality, relevant content, and technical support. Whereas course quality, relevant content, completeness of the content, attitudes toward students, and flexibility in taking courses were the five critical success factors from the students’ perspective. However, the focus in this paper was on E-learning during the COVID-19 pandemic, which is an abnormal situation. Thus, we implemented the AHP method in order to calculate the weight of the different criteria.

2.4. Critical Success Factors and Type of E-Learning System

The critical success factors are referred to as ”characteristics, conditions, or variables that, when properly sustained, maintained, or managed, can have a significant impact on the success of a firm competing in a particular industry” [22]. By finding the CSFs, stakeholders can boost these factors for better outcomes. The types of E-learning systems are identified in Table 1, to determine the most appropriate type of system associated with AHP and the TOPSIS technique during COVID-19. The factors that were evaluated in this paper are identified and defined in Table 2. Figure 1 summarizes the critical success factor hierarchy problem discussed in this paper based on the multiple-criteria decision analysis problem representation. Those criteria and alternatives were identified from previous studies on E-leaning and were categorized to represent the majority of the criteria that were evaluated in other studies under different conditions.

3. Materials and Methods

The methodology in this research consists of three parts; a survey, the AHP method, and the TOPSIS method. Further details will be provided in the next subsections.

3.1. Data Collection

Firstly, the research team visited 69 educational institutions and planned several meetings with the distance learning managers to create an appropriate picture of the problem. Table 3 summarizes the E-learning management staff demographic data in terms of age, gender, nationality, academic degree, and job title. This shows that most participants (90%) had a PhD, and that, with respect to job title, the biggest group were Associate Professors (45%). Furthermore, the majority were Saudi nationals (71%). The managers evaluated the criteria associated with each E-learning system. During the first meeting, the discussion of the decision criteria were as follows: Instructor Characteristics, Student Characteristics, Information Technology, Support, Technology Knowledge, Course, Instructional Design, E-Learning Environment, Level of Collaboration, and Knowledge Management. The decision alternatives were as follows: Blended Learning, Flipped Classroom, Information and Communication Technology (ICT) Supported, Face-to-Face Learning, Synchronous Learning, and Asynchronous Learning. The managers of E-learning rated each criterion associated with each e-learning system toward the CSFs of E-learning in the COVID-19 pandemic, as shown in Table 4.

3.2. The Analytic Hierarchy Process (AHP)

To apply the AHP method, the following steps were performed based on [19,33]
Step 1: Weight each criterion and decision alternatives. The ratings are given in Table 5.
For instant, if the E-learning managers determined that, in the E-learning process and based on the criteria and the rankings shown in Table 5, the Instructor Characteristics were very strongly preferable to the Student Characteristics (in other words, the E-learning Instructor Characteristics had a greater influence on the learning process compared with the Students Characteristics), they will give rank 7 to the Instructor Characteristics in the pair-wise comparison matrix when comparing the Instructor Characteristics versus Students Characteristics. On the other hand, the intermediate ratings (2, 4, 6, and 8) were also used. The reciprocal ratings (1/9, 1/8, etc.) were used in cases where a second alternative was chosen over the first, assigning a rating in the case of comparing an alternate with itself, as shown in Table 6. For example, Student Characteristics pair-wise compared to Instructor Characteristics at 1/7, which indicates that the Student Characteristics were seven times more preferable to Instructor Characteristics from the managerial perspective.
Step 2: Develop the pair-wise comparison matrix and rate the relative importance among every pair of decision alternatives, as shown in Table 6. The alternatives are listed in the matrix horizontally (first alternative) and vertically (second alternative) as associated with the numerical ratings.
Step 3: Set up the normalized matrix by dividing all numbers in the pair-wise comparison matrix column by the sum of its column, as shown in Table 7.
Step 4: Determine the average priority vector for all the rows in the normalized matrix. Then use these averages to create the priority vector of all alternative preferences associated with the criterion where the sum of this vector is 1, as shown in Table 8.
Step 5: Calculate the consistency ratio and use it to measure the subjective input. A ratio of less than 0.1 was considered good. For the ratios greater than 0.1, the subjective input was considered for re-evaluation.
Step 6: Develop the priority matrix to obtain the priority vectors for each criterion using the results of Step 4.
Step 7: Develop the criterion pair-wise development matrix following the one used to create the alternative pair-wise comparison matrices utilizing the ratings of Step 2, normalize the matrix following Step 3, and develop the priority vector criterion in Step 4.
Step 8: Develop the overall priority vector through multiplying the criteria priority vector in Step 7 with the priority matrix in Step 6.
Microsoft Excel was used to perform all these steps.

3.3. The Technique for Order Preference by Similarity to Ideal Solution (TOPSIS)

To apply the TOPSIS method, the following steps were performed, based on [32,34]
Step 1: Form the matrix expressed as follows:
D = [ A 1 X 1 X 2 X n A 2 X 11 X 12 X 1 n A 3 X 21 X 22 X 2 n A m X m 1 X m 2 X m n ]
where:
Ai = ith alternative project and Xij = the numerical outcome of the ith alternative project with respect to the jth criterion.
Table 9 shows the structure of the TOPSIS matrix.
Step 2: Normalize the decision matrix (D) by applying the following formula:
r i j = X i j i = 1 n X i j 2
Table 10 shows the TOPSIS normalize matrix.
Step 3: Construct the weighted normalized decision matrix that is shown in Table 11 by multiplying the normalized decision matrix by its relative weights. The following formula is used to calculate the weighted normalized value vij:
V i j = W i j R i j
Step 4: Define both the positive and negative ideal solutions:
A * = { ( m a x   v i j | j є J ) ,   ( m i n   v i j | j є J ) }     J = 1 ,   2 ,   3 ,   ,   n
A = { ( m i n   v i j | j є J ) ,   ( m a x   v i j | j є J ) }     J = 1 ,   2 ,   3 ,   ,   n
where:
J is associated with the benefit criteria and J′ is associated with the cost criteria.
Table 12 shows the TOPSIS positive and negative ideal solutions.
Step 5: Calculate the separation of all alternatives from the positive ideal as follows:
S i * = j = 1 n ( V i j V j * ) 2     i = 1 , 2 , , m
Step 6: Calculate the relative distance of Ai with respect to A*, the ideal solution, which is defined as:
C i * = S i ( S i * + S i ) ,   0 C * 1     i = 1 , 2 , , m
where performance alternatives become better with larger values of Ci*.
Step 7: Rank the order of preference.
All steps were performed using Microsoft Excel.

4. Results and Discussion

The data were collected by interviewing 69 E-learning managers. The response rate for both the first and second rounds of interviews was 100%. Every factor’s weight was calculated using AHP software and Microsoft Excel 2013. The inconsistency ratio was calculated to achieve consistent weights among all participants. Table 8 shows that Knowledge Management (0.347996), Support (0.169233), Student Characteristics (0.111681), and Information Technology (0.106554) were the most critical success factors that influenced the E-learning process during the COVID-19 pandemic.
The primary purpose of the TOPSIS technique is to discover the best alternative that should have the shortest distance, which is the Euclidean distance, from the ideal solution. The data for the E-learning system alternatives based on their criteria are provided in Table 9, which shows the calculation of the square root of the squared summation for the given criteria for each E-learning system. The value in every cell was divided by the root summation of the square value, which gives the normalized decision matrix, as shown in Table 10. Table 11 shows the weighted normalized decision matrix, along with the ideal best and the ideal worst values. Table 12 shows the Euclidean distance from the value of both the ideal best and worst along with the performance score for every alternative using the TOPSIS method.
The research team used Microsoft Excel 2013 software to analyze and employ both AHP and TOPSIS techniques. The demonstration of the steps and analysis was provided in Section 3. As the result of the analytic hierarchy process for the E-learning critical success factors during the COVID-19 pandemic, Table 12 shows that Blended Learning appeared to be the best decision alternative for educational institutions to consider when selecting an E-learning system during the COVID-19 pandemic, with a total weight of 0.811. This was followed by Asynchronous Learning, which was considered to be the second-best alternative, with a total weight of 0.564, and then by the ICT Supported Face-to-Face Learning with a total weight of 0.563. This was then followed by the Flipped Classroom with a total weight of 0.483 and, finally, Synchronous Learning with a total weight of 0.066.
The findings revealed that the course type and contents did not have such a great impact on learning outcomes as previous studies showed, because it is a managerial perspective and all the courses are quoted with the same weight/importance [35]. A focus on providing the users (students and instructors) with more training in knowledge management would be wise [36]. Educational institutional support is very important in the success of E-learning. This is compatible with [37], which stated that providing computer and training support to students positively impacted the students’ use of the learning system. Our study found that student characteristics played a large role in educational systems during COVID-19.
The findings also indicated that students must understand their role during the social distancing measures, build their own attitude and commitment, and find ways to self-motivate in order to gain successful learning outcomes. The findings for student characteristics are consistent with a study from [26,38].
The primary objective of this paper was to identify and prioritize the critical success factors of E-learning system adoption during the COVID-19 pandemic. We employed AHP and TOPSIS, and this study may assist educational institutions in gaining a better understanding of the critical success factors for E-learning adoption during the COVID-19 pandemic. Focusing on crises, such as the COVID-19 pandemic, the current study is relevant as no previous research has discussed issues where the entire educational system worldwide was affected with many interruptions. Many educational institutions shifted from in-class education toward E-learning. E-learning system adoption is not an easy process, nor can one system fit all different types of disciplines and institutions around the globe. This research tackles the different possible systems and their critical attainment factors.

5. Limitations

This research tackled the different possible systems and their critical attainment factors. There are many studies regarding general education and E-learning; however, there was a lack of literature that included the effect of pandemics that would reflect the same situation that the world is facing during COVID-19. One of the drawbacks of the multi-criteria decision analysis tools is that they are very sensitive to the perspective the study is focused on. Therefore, the same applied tools would provide different results and findings based on whether the issue was tackled from the student perspective or the instructor perspective. Finally, this study took place in Saudi Arabia, and might not apply to other countries due to differences in perspective and regulations.

6. Conclusions

The educational process worldwide has been interrupted due to the COVID-19 pandemic. E-learning is becoming much more necessary, and is very important in education. Educational institutions during COVID-19 face the unique challenges of smoothly maintaining the process of learning while ensuring that it is still beneficial. Therefore, these institutions must understand what drives instructors and learners toward the E-learning system. The main focus of this study was to classify and prioritize E-learning systems during the COVID-19 pandemic as well as to recognize practical implications.
This study prioritized different systems of E-learning using multi-criteria approaches. We discovered that the most significant factors influencing E-learning success during the COVID-19 pandemic were related to technology knowledge management, support from management, increased student awareness of utilizing E-learning systems, and demanding a high level of information technology from the instructors, students, and universities. This finding should be seriously considered as no matter how great the technology is, readiness for E-learning implementation still plays the leading role in improving the educational process. Blended Learning was the most preferred E-learning system out of the five methods discussed in this study. The results of this study provide useful information to the E-learning managers of universities in their process of implementing modern technologies in education.

Author Contributions

A.A.R. collected, processed, and analyzed the data and wrote the paper. A.Y.A. conceived and designed the study, defined the research subject, and directed the research from start to finish. He provided important advice throughout the study and helped in editing the manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Toth-Stub, S. Countries Face an Online Education Learning Curve: The Coronavirus Pandemic has Pushed Education Systems: Online, Testing Countries’ Abilities to Provide Quality Learning for All. 2020. Available online: https://www.usnews.com/news/best-countries/articles/2020-04-02/coronavirus-pandemic-tests-countries-abilities-to-create-effective-online-education (accessed on 27 April 2020).
  2. COVID-19 Educational Disruption and Response. 2020. Available online: https://en.unesco.org/covid19/educationresponse (accessed on 19 May 2020).
  3. World Health Organization. WHO Coronavirus Disease (COVID-19) Dashboard. December 2019. Available online: https://covid19.who.int/ (accessed on 22 May 2020).
  4. Graham, C.R.; Woodfield, W.; Harrison, J.B. A framework for institutional adoption and implementation of blended learning in higher education. Internet High. Educ. 2013, 18, 4–14. [Google Scholar] [CrossRef]
  5. World Health Organization. Coronavirus Disease (COVID-19) Advice for the Public (Advice for the Public). Available online: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/advice-for-public (accessed on 29 April 2020).
  6. Abbas, Z.; Umer, M.; Odeh, M.; McClatchey, R.; Ali, A.; Farooq, A. A semantic grid-based e-learning framework (SELF). In Proceedings of the CCGrid 2005. IEEE International Symposium on Cluster Computing and the Grid 2005, CWL, UK, 9–12 May 2005; Volume 1, pp. 11–18. [Google Scholar]
  7. Muhammad, A.; Ghalib, M.F.M.D.; Ahmad, F.; Naveed, Q.N.; Shah, A. A study to investigate state of ethical development in e-learning. J. Adv. Comput. Sci. Appl. 2016, 7, 284–290. [Google Scholar] [CrossRef]
  8. Naveed, Q.N.; Muhammad, A.; Sanober, S.; Qureshi, M.R.N.; Shah, A. A mixed method study for investigating critical success factors (CSFs) of e-learning in Saudi Arabian universities. Methods 2017, 8. [Google Scholar] [CrossRef]
  9. Hameed, S.; Badii, A.; Cullen, A.J. Effective e-learning integration with traditional learning in a blended learning environment. In Proceedings of the European and Mediterranean Conference on Information Systems, Al Bustan Rotana, Dubai, UAE, 25–26 May 2008; pp. 25–26. [Google Scholar]
  10. Basak, S.K.; Wotto, M.; Bélanger, P. A framework on the critical success factors of e-learning implementation in higher education: A review of the literature. Int. J. Educ. Pedagog. Sci. 2016, 10, 2409–2414. [Google Scholar]
  11. Al-Dosari, H. Faculty members and students perceptions of e-learning in the English department: A project evaluation. J. Soc. Sci. 2011, 7, 291. [Google Scholar] [CrossRef]
  12. Al-Asmari, A.M.; Khan, M.S.R. E-learning in Saudi Arabia: Past, present and future. Near Middle East. J. Res. Educ. 2014, 2014, 2. [Google Scholar] [CrossRef]
  13. Hwang, C.L.; Yoon, K. Multiple Attribute Decision Making: Methods and Applications; Springer: Berlin/Heidelberg, Germany, 1981. [Google Scholar]
  14. Beckmann, M.; Künzi, H.P.; Hwang, C.L.; Yoon, K. Multiple Attribute Decision Making; Scientific Research Publishing: Southern California, CA, USA, 1981; Volume 186. [Google Scholar]
  15. Prakash, C.; Barua, M.K. Integration of AHP-TOPSIS method for prioritizing the solutions of reverse logistics adoption to overcome its barriers under fuzzy environment. J. Manuf. Syst. 2015, 37, 599–615. [Google Scholar] [CrossRef]
  16. Mohammed, H.J.; Kasim, M.M.; Shaharanee, I.N. Evaluation of E-learning approaches using AHP-TOPSIS technique. J. Telecommun. Electron. Comput. Eng. (JTEC) 2018, 10, 7–10. [Google Scholar]
  17. Ince, M.; Yigit, T.; Isik, A.H. AHP-TOPSIS method for learning object metadata evaluation. Int. J. Inf. Educ. Technol. 2017, 7, 884–887. [Google Scholar] [CrossRef]
  18. Andayani, S.; HM, B.S.; Waryanto, N.H. Comparison of Promethee–Topsis method based on SAW and AHP weighting for school e-learning readiness evaluation. J. Phys. Conf. Ser. 2020, 1581, 012012. [Google Scholar] [CrossRef]
  19. Saaty, T.L. What is the analytic hierarchy process? In Mathematical Models for Decision Support; Springer: Berlin/Heidelberg, Germany, 1988; pp. 109–121. [Google Scholar]
  20. Dweiri, F.; Kumar, S.; Khan, S.A.; Jain, V. Designing an integrated AHP based decision support system for supplier selection in automotive industry. Expert Syst. Appl. 2016, 62, 273–283. [Google Scholar] [CrossRef]
  21. Anggrainingsih, R.; Umam, M.Z.; Setiadi, H. Determining e-learning success factor in higher education based on user perspective using Fuzzy AHP. MATEC Web Conf. 2018, 154, 03011. [Google Scholar] [CrossRef]
  22. Leidecker, J.K.; Bruno, A.V. Identifying and using critical success factors. Long Range Plan. 1984, 17, 23–32. [Google Scholar] [CrossRef]
  23. Thai, N.T.T.; De Wever, B.; Valcke, M. The impact of a flipped classroom design on learning performance in higher education: Looking for the best "blend" of lectures and guiding questions with feedback. Comput. Educ. 2017, 107, 113–126. [Google Scholar] [CrossRef]
  24. Young, T.P.; Bailey, C.J.; Guptill, M.; Thorp, A.W.; Thomas, T.L. The flipped classroom: A modality for mixed asynchronous and synchronous learning in a residency program. West. J. Emerg. Med. 2014, 15, 938. [Google Scholar] [CrossRef]
  25. Alhabeeb, A.; Rowley, J. E-learning critical success factors: Comparing perspectives from academic staff and students. Comput. Educ. 2018, 127, 1–12. [Google Scholar] [CrossRef]
  26. Muianga, X.; Klomsri, T.; Tedre, M.; Mutimucuio, I. From teacher-oriented to student-centred learning: Developing an ict-supported learning approach at the eduardo mondlane university, mozambique. Turk. Online J. Educ. Technol. 2018, 17, 46–54. [Google Scholar]
  27. Scholkmann, A. What I learn is what I like. How do students in ICT-supported problem-based learning rate the quality of the learning experience, and how does it relate to the acquisition of competences? Educ. Inf. Technol. 2017, 22, 2857–2870. [Google Scholar] [CrossRef]
  28. Rowe, J.A. Synchronous and Asynchronous Learning: How Online Supplemental Instruction Influences Academic Performance and Predicts Persistence. Ph.D. Thesis, Capella University, Minneapolis, MN, USA, 2019. [Google Scholar]
  29. Abdel-Gawad, T.; Woollard, J. Critical success factors for implementing classless e-learning systems in the Egyptian higher education. Int. J. Instr. Technol. Distance Learn. 2015, 12, 29–36. [Google Scholar]
  30. Alhabeeb, A.; Rowley, J. Critical success factors for eLearning in Saudi Arabian universities. Int. J. Educ. Manag. 2017, 31, 131–147. [Google Scholar] [CrossRef]
  31. Bhuasiri, W.; Xaymoungkhoun, O.; Zo, H.; Rho, J.J.; Ciganek, A.P. Critical success factors for e-learning in developing countries: A comparative analysis between ICT experts and faculty. Comput. Educ. 2012, 58, 843–855. [Google Scholar] [CrossRef]
  32. Behzadian, M.; Otaghsara, S.K.; Yazdani, M.; Ignatius, J. A state-of the-art survey of TOPSIS applications. Expert Syst. Appl. 2012, 39, 13051–13069. [Google Scholar] [CrossRef]
  33. Yang, J.; Lee, H. An AHP decision model for facility location selection. Facilities 1997, 15, 241–254. [Google Scholar] [CrossRef]
  34. Bhutia, P.W.; Phipon, R. Application of AHP and TOPSIS method for supplier selection problem. IOSR J. Eng. 2012, 2, 43–50. [Google Scholar] [CrossRef]
  35. Brophy, J. Teaching. Educational Practices Series—1; International Bureau of Education: Geneva, Switzerland, 2000. [Google Scholar]
  36. Miller, M.; Lu, M.Y.; Thammetar, T. The residual impact of information technology exportation on Thai higher education. Educ. Technol. Res. Dev. 2004, 92–96. [Google Scholar] [CrossRef]
  37. Lee, Y.C. The role of perceived resources in online learning adoption. Comput. Educ. 2008, 50, 1423–1438. [Google Scholar] [CrossRef]
  38. Muhammad, A.; Shaikh, A.; Naveed, Q.N.; Qureshi, M.R.N. Factors affecting academic integrity in e-learning of saudi arabian universities. An investigation using delphi and AHP. IEEE Access 2020, 8, 16259–16268. [Google Scholar] [CrossRef]
Figure 1. Critical success factors problem hierarchy.
Figure 1. Critical success factors problem hierarchy.
Education 10 00216 g001
Table 1. E-Learning system definition and prior research.
Table 1. E-Learning system definition and prior research.
Type of E-Learning SystemPrior ResearchDefinition
Blended Learning[4,16,23]Mix of traditional and online classes
Flipped Classroom[16,23,24]Student-centered approach with online material provided to students prior to classes
ICT Supported Face-to-Face Learning[25,26,27]Traditional learning supported by information and communication technology.
Synchronous Learning[16,24,28]A real-time interaction distance learning
Asynchronous Learning[16,24,28]Non-real time interaction distance learning
Table 2. E-Learning criteria, definition, and prior research.
Table 2. E-Learning criteria, definition, and prior research.
FactorsPrior ResearchDefinition
Student Characteristics[25,29,30,31]This factor focuses on the student’s environment while learning. It includes the student’s pace of learning, commitment, attitude, motivation, knowledge of computer systems, and demographics.
Instructor Characteristics[8,25,29,30,31]This factor focuses on the instructor’s environments while teaching. It includes the instructor’s attitude, flexibility, knowledge of learning technology, teaching style, and efficacy in student motivation.
Learning Environment[8,25,29,31] This factor focuses on the learning environment and facilities that are provided for both students and instructors. It includes a learning management system, technical infrastructure, interactive learning, and access and navigation.
Instructional Design[8,25,29,30]This factor focuses on the instructional system to meet the objectives of the institution. It includes the content quality, objective clarity, learning strategies, and learning psychology.
Support[8,25,29,30,31]This factor focuses on supporting both the instructors and students to enhance their experience. It includes communication tools, help disk availability, and training.
Information Technology[21,25,26,29,30,32]This factor focuses on the information technology system to deliver learning materials and objectives. It includes ease of use, reliability, efficiency, privacy, and information.
Technology Knowledge[8,25,29,30,31]This factor focuses on the knowledge of using technology for both instructors and students. It includes the use of computers, the use of software, and communication interaction.
Course[8,20,21,25,29,31]This factor focuses on the course material and objectives. It includes course evaluation, assessments, content development, and learning evaluations.
Level of Collaboration[8,21,30,31]This factor focuses on the collaboration level between faculty members. It includes the lack of social interaction, project team supervision, and managerial support.
Knowledge Management[8,21,30,31]This factor focuses on the management knowledge within the educational institution for faculty members and administration. It includes the management team, managing delivery and maintenance, time management, thinking strategies, and implementation expertise.
Table 3. E-Learning management staff demographic data.
Table 3. E-Learning management staff demographic data.
FrequencyPercentage
Age30–39 Years1725%
40–49 Years2435%
50–59 Years2841%
GenderMale5783%
Female1217%
NationalitySaudi4971%
Non-Saudi2029%
Academic DegreeMaster’s Degree710%
PhD6290%
Job TitleLecturer710%
Assistant Professor1928%
Associate Professor3145%
Professor1217%
DisciplineEducation710%
Science913%
Arts46%
Business2232%
Medicine23%
Engineering1928%
Political Science69%
Table 4. E-Learning approaches and evaluation criteria.
Table 4. E-Learning approaches and evaluation criteria.
No.AlternativesNo.Criteria
1Blended Learning1Instructor Characteristics
2Flipped Classroom2Student Characteristics
3ICT Supported Face-to-Face Learning3Information Technology
4Synchronous Learning4Support
5Asynchronous Learning5Technology Knowledge
6Course
7Instructional Design
8E-Learning Environment
9Level of Collaboration
10Knowledge Management
Table 5. Analytic Hierarchy Process (AHP) ratings.
Table 5. Analytic Hierarchy Process (AHP) ratings.
Linguistic Rating Numerical Rating
Extremely preferred 9
Very strongly preferred 7
Strongly preferred 5
Moderately preferred 3
Equally preferred 1
Table 6. Pair-wise comparison matrix.
Table 6. Pair-wise comparison matrix.
Instructor
Characteristics
Student CharacteristicsInformation TechnologySupportTechnology KnowledgeCourseInstructional DesignE-Learning
Environment
Level of
Collabor-Ation
Knowledge Management
Instructor Characteristics11/71/71/91/851/51/31/31/9
Student Characteristics7111/3553531/7
Information Technology7111/5353731/7
Support91/351775531/5
Technology Knowledge81/51/31/7151331/7
Course1/51/51/51/71/511/531/31/9
Instructional Design51/31/31/5151731/5
E-Learning Environment31/51/71/51/31/31/711/51/7
Level of Collaboration31/31/31/31/331/3511/5
Knowledge Management9775795751
Sum521115825451943222
Table 7. Normalized matrix.
Table 7. Normalized matrix.
Instructor
Characteristics
Student
Characteristics
Information
Technology
SupportTechnology
Knowledge
CourseInstructional
Design
E-Learning
Environment
Level of
Collaboration
Instructor
Characteristics
0.020.010.010.010.010.110.010.010.02
Student
Characteristics
0.130.090.060.040.20.110.160.120.14
Information
Technology
0.130.090.060.030.120.110.160.160.14
Support0.170.030.320.130.280.150.260.120.14
Technology
Knowledge
0.150.020.020.020.040.110.050.070.14
Course00.020.010.020.010.020.010.070.02
Instructional
Design
0.10.030.020.030.040.110.050.160.14
E-Learning
Environment
0.060.020.010.030.010.010.010.020.01
Level of
Collaboration
0.060.030.020.040.010.070.020.120.05
Knowledge
Management
0.170.650.450.650.280.20.260.160.23
Sum111111111
Table 8. Priority vectors.
Table 8. Priority vectors.
Instructor
Characteristics
Student CharacteristicsInformation TechnologySupportTechnology KnowledgeCourseInstructional DesignE-Learning EnvironmentLevel of CollaborationKnowledge Management
0.0251430.1116810.1065540.1692330.0681430.0225560.0760010.0231570.0495360.347996
Table 9. Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) matrix.
Table 9. Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) matrix.
Instructor CharacteristicsStudent CharacteristicsInformation TechnologySupportTechnology KnowledgeCourseInstructional DesignE-Learning EnvironmentLevel of Collaboration
Blended Learning1.7960.0030.0320.0060.436193.4880.490.496.452
Flipped Classroom2.1030.017000.194112.1480.0442250.001
ICT Supported Face-to-Face Learning0.8230.0040.1251.3280.162.4650.6860.65623.329
Synchronous Learning2.890.0520.2061.5950.0320.9220.240.2120.96
Asynchronous Learning2.62400.0060.0010.1410.4480.6890.08424.9
sum10.2350.0760.372.9310.962309.472.1454226.44255.642
√(sum)3.1990.2750.6081.7120.98117.5921.46465.0117.459
Table 10. Normalized matrix.
Table 10. Normalized matrix.
Instructor CharacteristicsStudent CharacteristicsInformation TechnologySupportTechnology KnowledgeCourseInstructional DesignE-Learning EnvironmentLevel of Collaboration
Blended Learning0.4190.1820.2960.0470.6730.7910.4780.0110.341
Flipped Classroom0.4530.4730.0330.0060.4490.6020.13710.004
ICT Supported Face-to-Face Learning0.2840.2210.5820.6730.4070.0890.5650.0120.648
Synchronous Learning0.531−0.832−0.747−0.7380.1840.0550.3350.0070.131
Asynchronous Learning0.5060.0450.1230.0190.3820.0380.5670.0040.669
sum2.1930.0880.2880.0072.0951.5752.0811.0351.792
Table 11. Weighted normalized decision matrix.
Table 11. Weighted normalized decision matrix.
Instructor CharacteristicsStudent CharacteristicsInformation TechnologySupportTechnology KnowledgeCourseInstructional DesignE-Learning EnvironmentLevel of Collaboration
Blended Learning0.01050.02030.03150.00790.04590.01780.03630.00020.0169
Flipped Classroom0.01140.05280.00350.0010.03060.01360.01040.02320.0002
ICT Supported Face-to-Face Learning0.00710.02470.0620.11390.02780.0020.0430.00030.0321
Synchronous Learning0.0134−0.093−0.0795−0.12490.01250.00120.02540.00020.0065
Asynchronous Learning0.01270.0050.01310.00330.02610.00090.04310.00010.0331
A*0.01340.05280.0620.11390.04590.01780.01040.00010.0002
A-0.0071−0.093−0.0795−0.12490.01250.00090.04310.02320.0331
Table 12. Priority vector.
Table 12. Priority vector.
Si*Si-Ci*Rank
Blended Learning0.11910.51080.81091
Flipped Classroom0.30630.28590.48274
ICT Supported Face-to-Face Learning0.27880.35860.56253
Synchronous Learning0.5620.03980.06625
Back to TopTop