Next Article in Journal
Impact of Gamified Teaching on University Student Learning
Previous Article in Journal
Collaboration Skills in Educational Robotics: A Methodological Approach—Results from Two Case Studies in Primary Schools
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Education Sciences
Volume 13
Issue 5
10.3390/educsci13050469
education-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

A Systematic Review of Critical Success Factors in Blended Learning

by
Wenhe Min
and
Zhonggen Yu
*
Faculty of Foreign Studies, Beijing Language and Culture University, Beijing 100083, China
*
Author to whom correspondence should be addressed.
Educ. Sci. 2023, 13(5), 469; https://doi.org/10.3390/educsci13050469
Submission received: 12 March 2023 / Revised: 5 April 2023 / Accepted: 10 April 2023 / Published: 3 May 2023
Browse Figures
Review Reports Versions Notes

Abstract

:
Against the backdrop of the post-pandemic period, there is an increasing need for blended learning in modern higher education systems. Critical success factors for blended learning should be considered as key indicators of learning outcomes. Therefore, the aim is to systematically review studies that examine the critical success factors for blended learning from the perspectives of the learner, instructor, course, design, technology, and environment. Eighty-two articles were analysed according to PRISMA (Preferred Reporting Items for Systematic Review and Meta-analysis) principles. The results show that critical success factors in these six dimensions have a positive impact on blended learning outcomes. These critical success factors are mainly learner characteristics, teacher characteristics, course materials and objectives, learning characteristics according to institutional objectives, ICT system, and learning environment. Future research could explore the impact of positive emotions on student and teacher learning outcomes in blended learning.

1. Introduction

With the proliferation and use of information and communication technologies (ICT), blended learning has become commonplace in modern higher education [1]. Most universities are integrating blended learning models into their teaching methods to enhance face-to-face learning. This has been testified by numerous studies that have examined the effect of online activities on the teaching and learning process [2]. Blended learning combines the characteristics of online and offline teaching and learning environments. In both environments, there are many important success factors that influence learning outcomes [3]. Much of the research to date has focused on the critical success factors of online learning during the pandemic. However, critical success factors may differ in the post-pandemic period from those in the pre-pandemic period [4]. Therefore, in this article we will focus on the critical success factors (CSF) for blended learning (BL) in the post-pandemic period.
As a pedagogical approach, blended learning combines face-to-face learning moments with online activities, thus facilitating a free and open dialogue between learners and teachers [5]. This combination has been shown to increase the interaction between learners and students, learners and teachers, and learners and content, thus maximising the effectiveness of learning [6]. As a result, blended teaching and learning approaches have developed exponentially in recent decades. However, online and offline learning have not been extensively studied in terms of key success factors. Moreover, previous studies have focused only on key variables that influence the success of e-learning, such as technology and learners (see Table 1) [3,7]. There is a need to develop a broader framework that integrates online and offline learning.
Although research on blended learning has grown considerably in recent years, there are relatively few studies that have conducted a systematic review of blended learning, and even fewer studies that have conducted a systematic review of the CSF of BL [12]. This study, therefore, aims to systematically examine the CSF of BL in six areas: Learners, Instructors, Course, Technology, Design and Environment, with more comprehensive implications for the development of blended learning (see Figure 1). The role of critical success factors in blended learning will be reflected in student performance and satisfaction.

2. Literature Review

2.1. Blended Learning

The concept of blended learning has undergone evolution since its appearance in the late 1990s [13]. Since 2000, ‘blended learning’ has mainly emphasised its physical characteristics. According to [14], blend learning combines two separate teaching modes, which are the online course and face-to-face course. At this stage, most scholars regard it as a transitional period between face-to-face teaching and online teaching. As more research has been conducted, blended learning has been applied as an independent teaching mode rather than a transitional one. On the one hand, blended learning has provided a clearer definition of the ratio between online and face-to-face teaching. On the other hand, scholars have begun to define blended learning from the perspective of teaching strategies, teaching methods, and teaching design in the blended learning context. Bliuc [15] thought that blended learning combined face-to-face interactions and technology-enhanced interactions between students, teachers, and learning resources. Later, blended learning changed from “a combination of online teaching and face-to-face teaching” to “a teaching environment based on the combination of mobile communication tools, network learning environment and classroom discussion” [15]. The current study mainly focused on the blended learning against the backdrop of the post-pandemic period, which integrated face-to-face teaching, network-based online teaching, technology-enhanced, and mobile learning.
Blended learning is a new kind of learning method which combines traditional face-to-face learning and online activities [16,17]. With the development of technology, blended learning is increasingly becoming one of the most promising teaching and learning methods in higher education [18]. Educators strongly recommend combining face-to-face teaching with online activities in order to alternate distance learning. Blended learning has many advantages, such as improved communication between teachers and students, effective collaboration between students, more student-centred learning, and greater flexibility in learning and teaching [19,20,21,22,23]. Although blended learning in higher education has already been researched, it should be noted that the CSFs of BL have not been extensively studied to date.

2.2. Satisfaction in Blended Learning

In the context of BL applications, satisfaction is defined as learners’ feelings and attitudes resulting from what they wish to obtain by interacting with the blended learning system [24]. Ref. [25] defines the concept of satisfaction as the final goal of any service, and according to [19], satisfaction is generally recognised as the measure of the quality and effectiveness of any type of teaching and learning [19]. In this study, learners’ satisfaction is considered as a subjective perception, i.e., students’ perceived satisfaction depends on the extent to which they consider that the BL programme achieves their desired results [26]. Since learners’ satisfaction is a subjective perception, more successful students are more satisfied than their peers [24].
Satisfaction plays an important role in the effectiveness of blended learning. Technology-enabled learning helps increase learners’ intrinsic motivation, which could further motivate satisfaction. Satisfaction helps learners to integrate mobile technology-enabled learning better than other negative feelings. It can also play a crucial role in the acceptance and effectiveness of mobile learning [27]. Thus, understanding the determinants of learner satisfaction provides insights for developing effective strategies that benefit learners. Therefore, this study uses the research model proposed in [28] to summarise the main factors affecting learner satisfaction in blended learning environments. This analysis assesses the impact of six parameters on learner satisfaction in BL environments.

2.3. Performance in Blended Learning

As an important measure of academic success, performance has a decisive influence on students’ development. In educational studies, researchers have used different theories to examine academic performance [18]. Acosta et al. [29] proposed a conceptual model of the relationships associated with academic success in the context of student characteristics and validated it in a BL environment. Ref. [30] examined the effects of student and environmental characteristics on BL in the context of previous research on factors influencing blended learning. The results showed that students’ behaviour during lessons is influenced by perceptions [31] and that these factors can be classified as design features, technology features, learner features, and instructor features [2]. Based on the previous studies, this article will provide a more comprehensive description of the CSFs that influence academic performance.

2.4. Critical Success Factors

Critical success factors (CSFs) are considered as the conditions or variables that can play a decisive role in the success of an industry [32]. By taking CSFs into account, stakeholders can use them to achieve better results [4]. In [33], the CSFs for online learning are divided into three categories: human factors, content and learning factors, and institutional factors. Other studies offer additional perspectives. McPherson and Nunes [34] classify the CSFs into the issue of leadership, culture, design, technology, and delivery [2]. Ref. [35] found that CSFs can be categorised on a macro level, with categories such as instructors’ leadership, support, and willingness. Based on previous research, this study classified the CSFs into six aspects, which were identified as shown in Table 2.

2.5. Purposed and Research Questions

The aim of this study is to systematically review and synthesise findings on the CSFs in BL in terms of learner, instructor, course, technology, design, and environment. We proposed the six questions: (1) What are the CSFs in terms of the learner in BL contexts? (2) What are the CSFs in terms of the instructor in BL contexts? (3) What are the CSFs in terms of the course in BL contexts? (4) What are the CSFs in terms of the technology in BL contexts? (5) What are the CSFs in terms of the design in BL contexts? (6) What are the CSFs in terms of the blended environments?

3. Research Methods

3.1. Research Designs

This study used a rapid approach to assess the evidence in the literature based on the Preferred Items for Systematic Review and Meta-Analysis Program (PRISMA-P) [43,44]. It took four steps to identify and summarise previous literature in order to gain a comprehensive understanding of the CSFs for BL. First, the researchers conducted a literature search on Web of Science based on the proposed research questions. Second, they used clustering and mapping techniques in VOSviewer to identify popular research topics and formulate several research questions. Third, the authors reviewed the articles according to the inclusion and exclusion criteria. Finally, the authors reviewed and summarised the literature to gain an overall understanding of the CSFs of BL.

3.2. Research Corpus

The researchers developed their search strategies and obtained literature through browsing Web of Science on 10 September 2022. There are various databases in Web of Science, including SCI-EXPENDED (1900–2022), SSCI (1998–2022), AandHCI (1998–2022), Conference Proceedings Citation IndexScience (CPCI-S, 1998–2022), Conference Proceedings Citation Index-Social Science & Humanities (CPCI-SSH, 1998–2022), Emerging Sources Citation Index (ESCI, 2017–2022), Current Chemical Reactions (CCR-EXPENDED, 1985–2022), and Index Chemicus (IC, 1993–2022). In this way, selection bias can be minimised, and the representativeness can be increased [45].
The researchers collected the literature by searching online databases. On 9 September 2022, the authors searched on the Web of Science databases and only found 7 results by keying in “critical success factor*” (topic) AND “blend*learn*”. Later, the researchers added more topics that are related to blended learning and critical success factors. On 10 September 2022, the authors searched on the Web of Science databases and found 3700 results by keying in “critical success factor*” OR “critical factor*” OR “factor*” (topic) and “blend* learn*” OR “online learn*” OR “offline learn*” OR “e-learning” (topic). The time ranged from January 2008 to 10th September 2022.
To select search directions from the collected literature, the researcher obtained the results (N = 3700) in plain text and read them with VOSviewer. The data were then interpreted with VOSviewer, selecting “co-occurrence” as the type of analysis, “all keywords” as the unit of analysis, and “full counting” as the counting method. The minimum number of keyword occurrences was set at 1. Two hundred and seventy eight keywords reached this threshold. Figure 2 presents an overview of the bibliographic network.
714 keywords were divided into 4 clusters. Cluster 1 contained 13 terms, such as blended learning, technology, course design, and satisfaction. Cluster 2 contained 12 items, such as courses, environment, performance, engagement, teachers, and students. Cluster 3 contained 10 items, such as online learning environment, education technology, motivation, tool, and collaborative learning. Cluster 4 contained 9 terms, e.g., e-learning, satisfaction, students, learning environment, technology acceptance, and user acceptance.
The researchers chose popular research topics based on a keyword list with the highest number of co-occurrence links. The link strengths of learner (N = 182), instructor (N = 131), course (N = 162), technology (N = 238), design (N = 159), environment (N = 139), satisfaction (N = 167), and performance (N = 148) were at the top. The item ‘blended learning’ also showed a strong link strength (N = 294). The CSFs will thus centre on these top items, coupled with blended learning. Therefore, this study will focus on the CSFs in BL in terms of learner, instructor, course, technology, design, and environment. At the same time, the learning outcomes of blended learning will be reflected in learners’ satisfaction and performance.

3.3. Inclusion and Exclusion Criteria

Based on a systematic review of articles and PRISMA-P, researchers included and excluded the collected literature. Studies were included if they (1) focused on the critical success factors of blended learning, (2) provided sufficient information for the study, (3) were written in English, and (4) contained convincing results. Studies were excluded if they were (1) duplicates, (2) irrelevant, (3) not relevant to the research question, (4) written in another language, (5) not full texts, and (6) not relevant to the education sector.

3.4. Study Selection

Two researchers independently analysed the collected literature according to formal inclusion and exclusion criteria (see Figure 3). Four steps were followed. First, 3700 publications were retrieved from Web of Science. After analysing the types of publications, the researchers excluded review articles (N = 152), those published online (N = 224), meeting papers (N = 19), social materials (N = 9), data papers (N = 5), editorial materials (N = 4), meeting abstracts (N = 3), and letters (N = 1). After reviewing the titles and abstracts, the researchers selected 1612 publications for full-text review. After assessing the relevance of the full-text publications, the researchers finally included 3558 publications in this systematic review. The Cohen’s Kappa value was 0.899 (See the result of Kappa test in the Supplementary Material for comprehensive analysis), which indicates high inter-rater reliability between the two researchers.

4. Results

We summarise the CSFs for BL in this part of the paper. To systematically identify critical success factors, we divide them into six categories: learner, instructor, course, technology, design, and environment. The learner dimension focuses on learner characteristics, including learner traits, attitudes, motivation, and cognitive and demographic characteristics. The instructor dimension focuses on the characteristics of teachers. The curriculum dimension focuses on course materials and objectives. The technical dimension focuses on the ICT systems for presenting course material and learning outcomes. The design dimension focuses on the pedagogical features consistent with the learning objectives of the institution. The environmental dimension focuses on the learning environment and the equipment used by students and teachers.
RQ1: What are the critical success factors in terms of the learner in blended learning contexts?
This study found that in the learner dimension, critical success factors mainly include student characteristics, learning speed, commitment, attitude, motivation, cognition, computer efficacy and experience, and demographics. Most empirical studies have shown that learner characteristics determine the success of blended learning initiatives [28,46,47]. Theories have been proposed to establish the relationship between intrinsic factors that influence learners’ performance. These include motivation, emotions, cognition, and meta-cognition [48]. The role of positive emotions is important in blended learning [29]. There is also evidence to show that learners need to understand their role in blended learning, shape their attitudes and engagement, and motivate themselves to achieve good learning outcomes.
Furthermore, it is argued that students’ knowledge of computer games has a great effect on BL. The study [48] suggests that providing students with additional training in knowledge management would be a wise decision to improve their academic performance. Indeed, training can help students become more confident, especially in their ability to apply learning technologies, and ultimately impacts on their academic performance [2]. Research has shown that providing computer and pedagogical support to students has a positive impact on the use of learning systems and academic performance [49]. Studies have found that students’ prior experiences, perceptions, knowledge, and motivation to learn also affect their learning. Therefore, it is important to pay attention to several factors related to learner characteristics [50].
RQ2: What are the critical success factors in terms of the instructor in blended learning contexts?
The CSFs on the instructor’s side are, above all, attitude, knowledge of computer systems, and teaching style. According to one study [4], the characteristics of the teacher are much better than those of the learner. In other words, the instructor’s characteristics had a stronger impact on the study compared with the students’ characteristics. Selim [38] found that teacher characteristics, supported by technological infrastructure and the university, can contribute significantly to the success of BL. Some studies have also examined the impact of teachers’ teaching style on student learning [51]. It has shown that teachers who use an interactive teaching style can effectively promote students’ engagement and attitude towards learning. In general, responsiveness, awareness, honesty, teaching style, encouragement of student interaction, technical skills, communication skills, and teacher quality are essential for successful blended learning [2].
Positive teacher attitudes have also been identified as a critical success factor. Ref. [49] found that teachers’ positive attitudes have a strong influence on the success of technology adoption and implementation. Matthew Myers and Halpin [52] identified teacher’s attitude as an important predictor of technology adoption, as it is related not only to knowledge and value of the technology, but also to the application of theoretical concepts in the classroom. Teachers who show their positive attitudes towards the application of technology are more likely to see its value and apply it in their teaching [53].
However, some of the publications present conflicting results on the impact of instructor involvement on students’ perceptions. Although previous studies suggest that timely instructor assistance and support can contribute to changes in students’ perceptions of learning, especially when students struggle with e-learning courses [54], the study of [2] found no evidence of the impact of instructor involvement on students’ perceptions. This result contrasts with many prior studies [30,38,55] that confirmed the positive relationship between instructor involvement and blended learning success.
RQ3: What are the critical success factors in terms of the course in blended learning contexts?
CSFs in the course aspect are course design, assessment, evaluation, content quality, and flexibility. Ease of use and flexibility are often the most valuable features when developing content for blended learning [56]. McDonald [40] argues that a standardised approach to e-learning can improve learning outcomes for learners. This is because e-learning offers most learners flexibility and convenience in completing modules [33], which can have a positive impact on learning outcomes. In addition to flexibility and convenience, the quality of content is also crucial to the success of e-learning [55]. According to Corlane et al. [2], learners emphasise the need to maintain quality and management practices. Therefore, course quality is one of the CSFs to be taken into account by teachers and learners.
RQ4: What are the critical success factors in terms of the technology in blended learning contexts?
CSFs in the technological dimension are, first and foremost, ease of use, quality, reliability, efficiency, privacy, information, and the use of software. According to [28], previous studies have identified important variables that contribute significantly to the success of e-learning. These factors focused mainly on technology [7]. One study [2] found that critical success factors that influence learners’ perceptions are: access to computers, availability of an online system or environment, self-efficacy in using computers and online learning, and perceived ease of use (PEU) and convenience. This finding suggests that actual application of a kind of technology is directly or indirectly influenced by the perceived usefulness and convenience of the system. Factors such as PEU and PU have been shown to be important predictors of learning outcomes [33,36,40]. In particular, the results confirm that the learner’s willingness to learn in an online environment depends on computer self-efficacy, while students’ attitudes towards it are influenced by PEU. Furthermore, computer self-efficacy, social norms, and system accessibility can have a great effect on PEU [2].
RQ5: What are the critical success factors in terms of the design in blended learning contexts?
CSFs in the design aspect are quality of content, clarity of purpose, teaching methods, learning strategies, psychology of learning, PU, and PEU. According to [7], the design aspect has the strongest positive relationship with learner satisfaction. Furthermore, this study shows that the design aspect is a critical success factor for learner satisfaction. It also shows that the design aspect preferred by the younger generation is the most important factor in a BL environment. In addition, many pedagogical approaches to developing learning outcomes have been identified. DeMarcos et al. [57] argue that social gamification as a pedagogical approach can improve academic performance and promote social interaction during practical tasks.
RQ6: What are the critical success factors in terms of the blended environments?
CSFs in the environmental dimension are learning management systems, technological infrastructure, interactivity, variety of assessments, system accessibility, and ease of navigation as supporting tools. According to [3], conscientiousness and learning management systems are significantly related to the success of a course in a BL environment. In addition, the use of learning management systems and tools has a positive influence on learner performance in a BL environment [58]. The study also found that discussion posts, peer interaction, and practice were identified as important factors influencing students’ academic performance in a BL environment [59]. What is more, ref. [59] shows that accessibility to information affects interaction with available information sources, which in turn affects academic performance.
However, the literature presents partially contradictory results on the impact of the environment on students’ learning outcomes. Although previous studies have shown that the use of learning management systems and tools has a positive impact on student academic outcomes in a BL environment, Zhu et al. [60] found there are no significant differences between them. Furthermore, no direct relationship was observed between technology infrastructure and learning outcomes [2]. Thus, the impact of blended environments on student learning outcomes requires further research.

5. Discussion

This article systematically reviews the existing literature on CSFs of BL from the perspectives of learner, instructor, course, design, technology, and environment. The critical success factors for learners are mainly their own characteristics. Self-efficacy, which is one of the characteristics of a learner, has the greatest direct impact on learning outcomes [61]. In fact, without the presence of a face-to-face teacher, students’ sense of self-efficacy will play the key role in academic achievement [62,63]. Many empirical studies have shown that self-efficacy is positively linked to engagement, satisfaction, motivation, and learning performance [64,65,66]. The results showed that students with low self-efficacy tend to experience anxiety while learning English [67]. Therefore, students need to use different intentional strategies to take responsibility for knowledge construction [62,67].
RQ2 aims to identify critical success factors for instructors. Research shows that teachers’ positive attitudes strongly influence the success of blended learning, which indicates that positive emotions are associated with positive outcomes [67]. This is because teachers’ positive attitudes include not only the knowledge and value of technology, but also the application of theoretical concepts in the classroom [52]. Teachers who have positive attitudes towards technology are more likely to recognise its value and use it in their teaching [53]. Therefore, it is important to investigate the impact of teachers’ practices and how these practices improve student learning outcomes.
RQ3 seeks to explore critical success factors in the course dimension. Convenience and flexibility are often important factors when designing content for blended learning. Indeed, e-learning offers most learners flexibility and convenience in completing tasks [33], which can have a positive impact on learning outcomes. In addition, e-learning offers flexibility and convenience in managing space and time, which are important constraints in learning. Intuitively, this can have a positive impact on learners and teachers. Therefore, learners and teachers should take full advantage of the convenience and flexibility of blended learning to achieve better learning outcomes.
RQ4 addresses the critical success factors in the design dimension, which focuses on the features of the learning experience that meet the institution’s objectives. Previous research has shown that only the design dimension is a CSF for learning satisfaction. Therefore, it is important to pay more attention to the design dimension in order to achieve higher satisfaction of young learners in BL environments. The frequency of using learning strategies, i.e., repetition, detail, organisation, and critical thinking, was statistically higher in the blended learning group [29]. Therefore, these learning strategies can be taken into account when designing courses. On the other hand, unsatisfactory perceptions may deter students from continuing their studies. Therefore, teachers must pay attention to the course design in order to avoid making students feel this way.
RQ5 seeks to explore the technological aspects of the critical success factors, focusing on the learning materials and objectives offered by ICT systems. Blended learning and technology-assisted learning opportunities are increasingly popular in education. Several technology integration frameworks have been developed to assess the extent and nature of this integration of learning technologies [67]. Thus, significant investments in technology are needed to improve learning and knowledge development.
RQ6 refers to the CSFs in the environmental dimension. One of the most important factors is the learning management system. The use of learning management systems and tools has a positive influence on academic success in a BL environment. In fact, learning management systems can be used as tools to support blended learning to help learners achieve learning outcomes. In turn, learners’ satisfaction with the services and the quality of the information systems and technical system directly affects their decision to continue using the learning management system. Therefore, from the perspective of student satisfaction, higher education should consider the influencing factors before implementing blended learning environments with learning management systems [68]. Thus, it has been suggested that a more comprehensive analysis of the learning environment can improve the understanding of student behaviour [2].
Although blended learning can provide numerous potential benefits, there are many barriers to its successful application into higher education. For example, there is evidence that blended learning has not been successful due to high technology costs, inadequate support processes, and lack of a clear business strategy. In addition, difficulties in adopting and implementing courses and adapting to a competitive environment are obvious barriers to the development of blended learning. In addition, low success and adoption rates are due to increased instructor training time, increased instructor responsibility and involvement, low user comfort levels, and potentially increased levels of user frustration, anxiety, and confusion [2].
To address the challenges of implementing and managing BL, it is essential to notice the determinants of blended learning success in order to determine teaching and learning strategies [69]. It is noted that a better understanding can facilitate the development of more appropriate teaching strategies, pedagogical methods, and curricula, leading to wider adoption. As classrooms become increasingly reliant on information systems, it becomes more and more important to find the factors that influence the success of their implementation [70]. Furthermore, identifying success factors is even more important in developing countries, which face additional challenges and limited resources that may hinder success.

6. Conclusions

This section consists of the major findings, limitations, and future research directions.
In order to improve blended learning, this study provides an overview of the CSFs of BL. From the learner’s perspective, the CSFs are mainly the learner’s learning characteristics, such as self-efficacy, motivation, and cognition. From the teacher’s perspective, the CSF factors focus on the pedagogical characteristics of the teacher, such as positive attitude and teaching style. In the course aspect, the CSFs focus on course materials and course objectives, which mainly include quality, accessibility, and flexibility. The design dimension focuses on pedagogical features, which mainly include teaching methods and learning strategies. In the technological dimension, critical success factors focus on pedagogical features that meet institutional objectives, including ICT systems. Finally, critical success factors for the environment mainly include learning management systems, interoperability, and accessibility of systems.
Several limitations of this study should be noted here. Firstly, the study may not cover all relevant publications due to limited library resources. Secondly, the concept of blended learning has been broadened. This study focuses on blended learning in the context of ICT-supported face-to-face teaching. The inclusion of flipped classrooms, and of synchronous and asynchronous learning, would make it more comprehensive. Thirdly, there is a reciprocal relationship between the studied CSFs. However, this study only focuses on the impact of the CSFs on BL, while neglecting the impact of BL on CSF.
Blended learning should be seen as an important trend for the future of education. One of the most important findings of this study is that teachers’ positive attitudes have a strong influence on the success of technology implementation and adaptation. Therefore, researchers can further study how positive teacher attitudes can influence learning outcomes. Future research could be conducted in the enlarged blended learning context. In addition, the role of positive emotions in blended learning is also important. Therefore, future research could aim to further explore positive emotions influencing students’ learning outcomes.
Blended learning has been commonly used in higher education, especially with the introduction of learning management systems. This study has shown that the use of learning management systems in blended learning is positively related to students’ academic performance. Therefore, future research could examine the impact of CSFs on student learning outcomes and satisfaction with learning management systems in a BL environment. The study could also identify key indicators of the success of learning management systems.
Future research could focus on the effect of blended environments on student’ performance. Some publications present contradictory results on the effect of the environment on student learning performance While previous studies have shown that the use of learning management systems and tools is beneficial to learning outcomes in a BL environment, one study [2] suggests that there is no direct relationship between technological infrastructure and learning outcomes. Therefore, the effect of BL environments on student learning outcomes should be further investigated. In addition, future research could explore the design of BL environments to enhance student learning outcomes.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/educsci13050469/s1. A PRISMA checklist and a PRISMA abstract checklist [44] were provided as the supplementary materials to locate the checklist items in each section. Besides, the result of Kappa test was provided to enhance the reliability of study selection in Supplementary File S3.4. Moreover, the Title page was also provided in the supplementary materials.

Author Contributions

W.M.: Methodology, Investigation, Editing, and Writing—Original Draft; Z.Y.: Conceptualisation and Funding acquisition. All authors have read and agreed to the published version of the manuscript.

Funding

This work is supported by 2019 MOOC of Beijing Language and Culture University (MOOC201902) (Important) “Introduction to Linguistics”; “Introduction to Linguistics” of online and offline mixed courses in Beijing Language and Culture University in 2020; Special fund of Beijing Co-construction Project-Research and reform of the “Undergraduate Teaching Reform and Innovation Project” of Beijing higher education in 2020-innovative “multilingual +” excellent talent training system (202010032003); The research project of Graduate Students of Beijing Language and Culture University “Xi Jinping: The Governance of China” (SJTS202108).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

We make sure that all data and materials support our published claims and comply with field standards.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Ghazal, S.; Al-Samarraie, H.; Aldowah, H. “I am Still Learning”: Modeling LMS Critical Success Factors for Promoting Students’ Experience and Satisfaction in a Blended Learning Environment. IEEE Access 2018, 6, 77179–77201. [Google Scholar] [CrossRef]
  2. Barclay, C.; Donalds, C.; Osei-Bryson, K.-M. Investigating critical success factors in online learning environments in higher education systems in the Caribbean. Inf. Technol. Dev. 2018, 24, 582–611. [Google Scholar] [CrossRef]
  3. Alkis, N.; Temizel, T.T. The Impact of Motivation and Personality on Academic Performance in Online and Blended Learning Environments. Educ. Technol. Soc. 2018, 21, 35–47. [Google Scholar]
  4. Alqahtani, A.Y.; Rajkhan, A.A. E-Learning Critical Success Factors during the COVID-19 Pandemic: A Comprehensive Analysis of E-Learning Managerial Perspectives. Educ. Sci. 2020, 10, 216. [Google Scholar] [CrossRef]
  5. Law, K.M.Y.; Geng, S.; Li, T. Student enrollment, motivation and learning performance in a blended learning environment: The mediating effects of social, teaching, and cognitive presence. Comput. Educ. 2019, 136, 1–12. [Google Scholar] [CrossRef]
  6. Dziuban, C.D.; Hartman, J.L.; Moskal, P.D. Blended learning. Educause Cent. Appl. Res. Bull. 2004, 7, 12. [Google Scholar]
  7. Chen, W.S.; Yao, A.Y.T. An Empirical Evaluation of Critical Factors Influencing Learner Satisfaction in Blended Learning: A Pilot Study. Univers. J. Educ. Res. 2016, 4, 1667–1671. [Google Scholar] [CrossRef]
  8. Reich, A.; Rooney, D.; Lizier, A.L. Using technology integration frameworks in vocational education and training. Int. J. Train. Res. 2021, 19, 93–106. [Google Scholar] [CrossRef]
  9. Abdullah, F.; Ward, R. Developing a general extended technology acceptance model for e-learning (GETAMEL) by analysing commonly used external factors. Comput. Hum. Behav. 2016, 56, 238–256. [Google Scholar] [CrossRef]
  10. Song, Y.; Kong, S.C. Affordances and constraints of BYOD (Bring Your Own Device) for learning and teaching in higher education: Teachers’ perspectives. Internet High. Educ. 2017, 32, 39–46. [Google Scholar] [CrossRef]
  11. Fortier, M.S.; Vallerand, R.J.; Guay, F. Academic motivation and school performance: Toward a structural model. Contemp. Educ. Psychol. 1995, 20, 257–274. [Google Scholar] [CrossRef]
  12. Anthony, B.; Kamaludin, A.; Romli, A.; Raffei, A.F.M.; Phon, D.N.A.E.; Abdullah, A.; Ming, G.L. Blended Learning Adoption and Implementation in Higher Education: A Theoretical and Systematic Review. Technol. Knowl. Learn. 2022, 27, 531–578. [Google Scholar] [CrossRef]
  13. Feng, X.; Wang, R.; Wu, J. A Literature Review on Blended Learning: Based on Analytical Framework of Blended Learning. J. Distance Educ. 2018, 36, 13–24. [Google Scholar]
  14. Allen, I.E.; Seaman, J. Sizing the Opportunity: The Quality and Extent of Online Education in the United States, 2002 and 2003. Sloan Consort. 2003, 659–673. [Google Scholar]
  15. Bliuc, A.M.; Goodyear, P.; Ellis, R.A. Research Focus and Methodological Choices in Studies into Students’ Experiences of Blended Learning in Higher Education. Internet High. Educ. 2007, 4, 231–244. [Google Scholar] [CrossRef]
  16. Wasoh, F. Exploring the roles of blended learning as an approach to improve teaching and learning English. In Proceedings of the Multidisciplinary Academic Conference 2016, Albena, Bulgaria, 24–30 August 2016; pp. 165–171. [Google Scholar]
  17. Graham, C.R.; Woodfield, W.; Harrison, B. A framework for institutional adoption and implementation of blended learning in higher education. Internet High. Educ. 2013, 18, 4–14. [Google Scholar] [CrossRef]
  18. Yu, Z.; Xu, W.; Sukjairungwattana, P. Meta-analyses of differences in blended and traditional learning outcomes and students’ attitudes. Front. Psychol. 2022, 13, 926947. [Google Scholar] [CrossRef]
  19. Halverson, L.R.; Graham, C.R.; Spring, K.J.; Drysdale, J.S.; Henrie, C.R. A thematic analysis of the most highly cited scholarship in the first decade of blended learning research. Internet High. Educ. 2014, 20, 20–34. [Google Scholar] [CrossRef]
  20. Wu, J.; Tennyson, R.D.; Hsia, T.L. A study of student satisfaction in a blended e-learning system environment. Comput. Educ. 2010, 55, 155–164. [Google Scholar] [CrossRef]
  21. Garrison, D.R.; Kanuka, H. Blended learning: Uncovering its transformative potential in higher education. Internet High. Educ. 2004, 7, 95–105. [Google Scholar] [CrossRef]
  22. Garrison, D.; Vaughan, N.D. Blended Learning in Higher Education: Framework, Principles and Guidelines; Jossey-Bass: San Francisco, CA, USA, 2008. [Google Scholar]
  23. Chew, E. A Blended Learning Model in Higher Education: A Comparative Study of Blended Learning in UK and Malaysia. Unpublished Thesis, University of Glamorgan, Wales, UK, 2009. [Google Scholar]
  24. Diep, A.N.; Zhu, C.; Struyven, K.; Blieck, Y. Who or what contributes to student satisfaction in different blended learning modalities? Br. J. Educ. Technol. 2017, 48, 473–489. [Google Scholar] [CrossRef]
  25. Lin, Y.M.; Lin, G.Y.; Laffey, J.M. Building A Social and Motivational Framework For Understanding Satisfaction In Online Learning. J. Educ. Comput. Res. 2008, 38, 1–27. [Google Scholar] [CrossRef]
  26. Lo, C.C. How student satisfaction factors affect perceived learning. J. Scholarsh. Teach. Learn. 2010, 10, 47–54. [Google Scholar]
  27. Yu, Z.G.; Zhu, Y.; Yang, Z.; Chen, W. Student satisfaction, learning outcomes, and cognitive loads with a mobile learning platform. Comput. Assist. Lang. Learn. 2019, 32, 323–341. [Google Scholar] [CrossRef]
  28. Sun, P.C.; Tsai, R.J.; Finger, G.; Chen, Y.Y.; Yeh, D. What drives a successful e-learning? An empirical investigation of the critical factors influencing learner satisfaction. Comput. Educ. 2008, 50, 1183–1202. [Google Scholar] [CrossRef]
  29. Acosta-Gonzaga, E.; Ramirez-Arellano, A. The Influence of Motivation, Emotions, Cognition, and Metacognition on Students’ Learning Performance: A Comparative Study in Higher Education in Blended and Traditional Contexts. SAGE Open 2021, 11, 21582440211027561. [Google Scholar] [CrossRef]
  30. Naveed, Q.N.; Muhammad, A.; Sanober, S.; Qureshi, M.R.N.; Shah, A. A mixed method study for investigating critical success factors (CSFs) of e-learning in Saudi Arabian universities. Methods 2017, 8, 171–178. [Google Scholar]
  31. Otter, R.R.; Seipel, S.; Graeff, T.; Alexander, B.; Boraiko, C.; Gray, J.; Petersen, K.; Sadler, K. Comparing student and faculty perceptions of online and traditional courses. Internet High. Educ. 2013, 19, 27–35. [Google Scholar] [CrossRef]
  32. Leidecker, J.K.; Bruno, A.V. Identifying and using critical success factors. Long Range Plan. 1984, 17, 23–32. [Google Scholar] [CrossRef]
  33. Yoo, S.J.; Huang, W.-H.; Lee, D.Y. The impact of employee’s perception of organizational climate on their technology acceptance toward e-learning in South Korea. Knowl. Manag. E-Learn. Int. J. 2012, 4, 359–378. [Google Scholar]
  34. McPherson, M.A.; Nunes, J.M. Critical issues for e-learning delivery: What may seem obvious is not always put into practice. J. Comput. Assist. Learn. 2008, 24, 433–445. [Google Scholar] [CrossRef]
  35. Joo, Y.J.; Lim, K.Y.; Park, S.Y. Investigating the structural relationships among organisational support, learning flow, learners’ satisfaction and learning transfer in corporate e-learning. Br. J. Educ. Technol. 2011, 42, 973–984. [Google Scholar] [CrossRef]
  36. Martins, L.; Kellermanns, F.W. Student acceptance of a web-based course management system. Acad. Manag. Learn. Educ. 2004, 3, 7–26. [Google Scholar] [CrossRef]
  37. Selim, H.M. An empirical investigation of student acceptance of course websites. Comput. Educ. 2003, 40, 343–360. [Google Scholar] [CrossRef]
  38. Ong, C.-S.; Lai, J.-Y.; Wang, Y.-S. Factors affecting engineers’ acceptance of asynchronous e-learning systems in high-tech companies. Inf. Manag. 2004, 41, 795–804. [Google Scholar] [CrossRef]
  39. Selim, H.M. Critical success factors for E-learning acceptance: Confirmatory factor models. Comput. Educ. 2007, 49, 396–413. [Google Scholar]
  40. McDonald, D. Improved training methods through the use of multimedia technology. J. Comp. Inf. Syst. 1999, 40, 17–20. [Google Scholar]
  41. Abdel-Gawad, T.; Woollard, J. Critical success factors for implementing classless e-learning systems in the Egyptian higher education. Int. J. Instr. Technol. Distance Learn. 2015, 12, 29–36. [Google Scholar]
  42. Alhabeeb, A.; Rowley, J. E-learning critical success factors: Comparing perspectives from academic staff and students. Comput. Educ. 2018, 127, 1–12. [Google Scholar] [CrossRef]
  43. Crawford, C.; Boyd, C.; Jain, S.; Khorsan, R.; Jonas, W. Rapid evidence assessment of the literature (REAL): Streamlining the systematic review process and creating utility for evidence-based health care. BMC Res. Notes 2015, 8, 631. [Google Scholar] [CrossRef]
  44. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Syst. Rev. 2021, 10, 1–11. [Google Scholar] [CrossRef] [PubMed]
  45. Yu, Z.G.; Xu, W.; Yu, L. Constructing an online sustainable educational model in the COVID-19 pandemic environments. Sustainability 2022, 14, 3598. [Google Scholar] [CrossRef]
  46. Chang, C.-T.; Hajiyev, J.; Su, C.-R. Examining the students’ behavioral intention to use E-learning in Azerbaijan? The general extended technology acceptance model for E-learning approach. Comput. Educ. 2017, 111, 128–143. [Google Scholar] [CrossRef]
  47. Liaw, S.-S.; Huang, H.-M.; Chen, G.-D. Surveying instructor and learner attitudes toward E-learning. Comput. Educ. 2007, 49, 1066–1080. [Google Scholar] [CrossRef]
  48. Anderman, E.M.; Dawson, H. Learning with motivation. In Handbook of Research on Learning and Instruction; Mayer, R.E., Alexander, P.A., Eds.; Routledge: New York, NY, USA, 2011; pp. 219–241. [Google Scholar]
  49. Lee, Y.C. The role of perceived resources in online learning adoption. Comput. Educ. 2008, 50, 1423–1438. [Google Scholar] [CrossRef]
  50. Entwistle, N.J.; Peterson, E.R. Conceptions of learning and knowledge in higher education: Relationships with study behaviour and influences of learning environments. Int. J. Educ. Res. 2004, 41, 407–428. [Google Scholar] [CrossRef]
  51. Webster, J.; Hackley, P. Teaching effectiveness in technology mediated distance learning. Acad. Manage. J. 1997, 40, 1282–1309. [Google Scholar] [CrossRef]
  52. Myers, J.M.; Halpin, R. Teachers’ attitudes and use of multimedia technology in the classroom: Constructivist-based professional development training for school districts. J. Computer. Teacher Educ. 2002, 18, 133–140. [Google Scholar]
  53. Holden, H.; Rada, R. Understanding the influence of perceived usability and technology self-efficacy on teachers. J. Res. Technol. Educ. Technol. 2011, 43, 343–367. [Google Scholar] [CrossRef]
  54. Levy, Y.; Ramim, M.M. The E-learning skills gap study: Initial results of skills desired for persistence and success in online engineering and computing courses. In Proceeding of the Chais 2017 Conference on Innovative and Learning Technologies Research, Bologna, Italy, 26 June–1 July 2017; Association for Computing Machinery: New York, NY, USA, 2017; pp. 57–68. [Google Scholar]
  55. Ozkan, S.; Koseler, R. Multi-dimensional students’ evaluation of e-learning systems in the higher education context: An empirical investigation. Comput. Educ. 2009, 53, 1285–1296. [Google Scholar] [CrossRef]
  56. Arbaugh, J.B.; Duray, R. Class section size, perceived classroom characteristics, instructor experience, and student learning and satisfaction with web-based courses: A study and comparison of two online MBA programs. In Academy of Management Best Papers Proceedings (pp. A1–A6) [CD-ROM]; Nagao, D., Ed.; Academy of Management: Briarcliff Manor, NY, USA, 2001. [Google Scholar]
  57. De-Marcos, L.; García-Cabo, A.; López, E.G. Towards the Social Gamification of e-Learning: A Practical Experiment. Int. J. Eng. Educ. 2017, 33, 66–73. [Google Scholar]
  58. Baragash, R.S.; Al-Samarraie, H. Blended learning: Investigating the influence of engagement in multiple learning delivery modes on students’ performance. Telemat. Inform. 2018, 35, 2082–2098. [Google Scholar] [CrossRef]
  59. Mwalumbwe, I.; Mtebe, J.S. Using Learning Analytics to Predict Students’ Performance In Moodle Learning Management System: A Case Of Mbeya University Of Science And Technology. J. Inf. Syst. Dev. Ctries. 2017, 79, 1–13. [Google Scholar] [CrossRef]
  60. Vo, H.M.; Zhu, C.; Diep, N.A. The effect of blended learning on student performance at course-level in higher education: A meta-analysis. Stud. Educ. Eval. 2017, 53, 17–28. [Google Scholar] [CrossRef]
  61. Maini, R.; Sehgal, S.; Agrawal, G. Todays’ digital natives: An exploratory study on students’ engagement and satisfaction towards virtual classes amid COVID-19 pandemic. Int. J. Inf. Learn. Technol. 2021, 38, 454–472. [Google Scholar] [CrossRef]
  62. Tsai, C.; Shen, P.; Tsai, M. Developing an appropriate design of blended learning with web-enabled self-regulated learning to enhance students’ learning and thoughts regarding online learning. Behav. Inf. Technol. 2011, 30, 261–271. [Google Scholar] [CrossRef]
  63. Korr, J.; Derwin, E.B.; Greene, K.; Sokoloff, W. Transitioning an adult serving university to a blended learning model. J. Contin. High. Educ. 2012, 60, 2–11. [Google Scholar] [CrossRef]
  64. Liu, W.Y.; Yu, H.J. Effectiveness study of English learning in blended learning environment. Theory Pract. Lang. Stud. 2012, 2, 524–530. [Google Scholar] [CrossRef]
  65. Alsowat, H. An EFL flipped classroom teaching model: Effects on English language higher-order thinking skills, student engagement and satisfaction. J. Educ. Pract. 2016, 7, 108–121. [Google Scholar]
  66. Hu, J.; Peng, Y.; Chen, X.; Yu, H. Differentiating the learning styles of college students in different disciplines in a college English blended learning setting. PLoS ONE 2021, 16, e0251545. [Google Scholar] [CrossRef]
  67. Pappamihiel, N.E. English as a second language students and English language anxiety: Issues in the mainstream classroom. Res. Teach. Engl. 2002, 36, 327–355. [Google Scholar]
  68. Abidin, Z.; Rokhman, F.; Mathrani, A. Exploring the influencing factors of learning management systems continuance intention in a blended learning environment. Int. J. Innov. Learn. 2021, 30, 175–187. [Google Scholar] [CrossRef]
  69. Aparicio, M.; Bacao, F.; Oliveira, T. An e-learning theoretical framework. J. Educ. Technol. Soc. 2016, 19, 292. [Google Scholar]
  70. Yi, M.; Hwang, Y. Predicting the use of web-based information systems: Self-efficacy, enjoyment, learning goal orientation, and the technology acceptance model. Int. J. Hum. Comput. Stud. 2003, 59, 431–449. [Google Scholar] [CrossRef]
Education 13 00469 g001 550
Figure 1. A flowchart of the research process.
Figure 1. A flowchart of the research process.
Education 13 00469 g001
Education 13 00469 g002 550
Figure 2. A bibliographic network based on co-occurrence analysis.
Figure 2. A bibliographic network based on co-occurrence analysis.
Education 13 00469 g002
Education 13 00469 g003 550
Figure 3. A flow diagram of the study selection based on PRISMA.
Figure 3. A flow diagram of the study selection based on PRISMA.
Education 13 00469 g003
Table
Table 1. Previous reviews of critical success factors.
Table 1. Previous reviews of critical success factors.
StudiesStudy DimensionsFindingsContext
[7]1 dimension (technology)satisfactionBlended learning
[3]1 dimension (learner)Satisfaction and performanceOnline and blended learning
[7]1 dimension (design)performanceBlended learning
[8]1 dimension (technology)performanceIT technologies in the process of gradual study
[3,9]1 dimension (environment)Performance and engagementtraditional offline courses, the online film clip
[10]1 dimension (course)PerformanceBlended learning
[11]3 dimensions (learner, instructor, technology)perceptionOnline learning
The current studyAll dimensions included in selected studiesPerformance, satisfactionBlended learning
Table
Table 2. Critical Success Factors.
Table 2. Critical Success Factors.
FactorsPrior ResearchDefinition
learner[9,10,36,37,38]It mainly focuses on learner characteristics such as learning speed, interest, attitude, motivation, cognition, use of computer systems, computer efficiency and experience, and demographic characteristics.
instructor[31,33,39]It mainly focuses on the instructor’s teaching qualities, such as attitude, flexibility, responsiveness, knowledge of computer systems, teaching style and effectiveness.
course[30,40,41,42]It mainly focuses on course material and purpose, including course design, assessment, grading, quality of content, and flexibility.
design[30,41,42]It mainly focuses on instructional characteristics to align with the objectives of the institution, which include the clarity of the objective, teaching methods, learning strategies and psychology, perceived usefulness, and perceived ease of conducting.
technology[30,41,42]It mainly focuses on the ICT system to present learning resources and purpose, which include ease of use, quality, reliability, efficiency, privacy, information, and the use of the software.
environment[30,41,42]It mainly refers to the learning environment, including learning management systems, technological infrastructure, interactivity, variety of assessments, system accessibility, and ease of navigation and other facilitating conditions.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Min, W.; Yu, Z. A Systematic Review of Critical Success Factors in Blended Learning. Educ. Sci. 2023, 13, 469. https://doi.org/10.3390/educsci13050469

AMA Style

Min W, Yu Z. A Systematic Review of Critical Success Factors in Blended Learning. Education Sciences. 2023; 13(5):469. https://doi.org/10.3390/educsci13050469

Chicago/Turabian Style

Min, Wenhe, and Zhonggen Yu. 2023. "A Systematic Review of Critical Success Factors in Blended Learning" Education Sciences 13, no. 5: 469. https://doi.org/10.3390/educsci13050469

APA Style

Min, W., & Yu, Z. (2023). A Systematic Review of Critical Success Factors in Blended Learning. Education Sciences, 13(5), 469. https://doi.org/10.3390/educsci13050469

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop