Effects of Flipped Classroom on Learning Outcomes and Satisfaction: An Experiential Learning Perspective

Abstract

This article explores the flipped-classroom learning cycle and traditional blended-learning cycle from the experiential learning-theory perspective, according to which we proposed three teaching strategies: high-flipped classroom (high-FC), low-flipped classroom (low-FC), and Traditional Blended Learning (traditional BL). And, the three teaching strategies are adopted in the course of Enterprise Resource Planning System. The purpose is to investigate the effects of learning outcomes and satisfaction among high-FC, traditional BL, and low-FC. This research adopts a quasi-experimental method. The sample for the study is a compulsory course of the Information Management Department of the University of Science and Technology in Taiwan, with a total of 94 students in two classes. There are several research conclusions. (1) The three groups of t test results found that there are significant differences in the learning outcomes of the system skills. high-FC is better than low-FC, low-FC is better than traditional BL, and high-FC is better than traditional BL. However, there are no significant differences in the learning outcomes of process knowledge. (2) The results of ANOVA testing showed that students have the highest perceived learning outcomes for high-FC, followed by low-FC and traditional BL. (3) The results of ANOVA testing showed that students have the highest perceived learning satisfaction in high-FC, followed by low-FC, and the lowest is traditional BL. (4) The t test result shows that students are more satisfied with preview e-learning than review e-learning. The study findings provide several implications. (1) It is feasible to implement flipped classrooms from the perspective of experiential-learning theory. (2) Fully flipped classrooms have practical difficulties. A partially flipped classroom (high-FC or low-FC) can be a feasible gradual strategy. (3) Flipped classrooms and e-learning support each other’s sustainable development. (4) The flipped classroom based on experiential learning theoretical perspective has obtained preliminary verification in the ERP-system course.

1. Introduction

The Enterprise Resource Planning (ERP) system is the core system of contemporary enterprise management [1,2]. Several universities offer ERP-system courses to cultivate talent [2]. The content of ERP courses is diverse, not only the ERP system but also process knowledge [3,4]. In addition, students have no practical experience and lack opportunities to use the ERP system, which makes it difficult for students to learn the ERP system [4,5,6]. However, traditional teaching cannot bring ideal learning outcomes [7]. Educational theory and practice methods need to be applied to research in this area [7,8,9].

Dewey emphasized, “learning by doing” [10]. Experiential-learning theory (ELT) emphasizes that people learn through their observations and experiences [11,12,13]. Experiential learning plays an important role in the development and progress of education. Many higher-education courses obtain good results from employing experiential learning [14,15]. Some ERP studies have shown that ERP courses can benefit from experiential learning and improve student participation, learning outcomes, and satisfaction [7,9,16].

Jonathan Bergmann and Aaron Sams proposed the flipped classroom model flipping both teacher’s lecture and students’ homework. They allowed students to watch the video of lecture at home. Thus, they can actively participate in the class [17,18]. The flipped classroom combines pre-class e-learning and in-class active learning, changing learning from being teacher-oriented to student-oriented [19,20]. The flipped classroom can improve academic performance [21,22], satisfaction [23,24], engagement [25,26], learning motivation [27,28] etc. However, literature indicated diverse findings in terms of academic performance [29,30]. It came could be, several flipped classrooms contain a lack of a solid theoretical foundation, and some flipped classroom study studies completely based on educational theory [19,31,32].

The ELT emphasizes learning through experience and practice. The flipped classroom (FL) emphasizes completing tasks in the classroom. There is a close relationship between experiential learning and flipped classrooms, in practice [19]—both emphasize the importance of exercising or active learning. The flipped classroom, based on the ELT perspective, is worthy of exploration as experiential learning through ERP-system courses can improve learning outcomes [7,9,16,33,34]. However, there is still a lack of flipped classroom research on ERP-system courses. The primary focus of the study is the ELT-flipped classroom. We investigate learning outcomes and satisfaction between traditional blended-learning and ELT-flipped classrooms.

We distinguish between traditional blended-learning and flipped classrooms based on the teacher’s lecture time, students’ practice time in class, and the sequence of lectures conducted through video. (1) In traditional blended learning (traditional BL), teachers use most of their time to lecture, and students have very little time to practice, and video lectures are provided after class, for review. (2) In a low-flipped classroom (low-FC), the teacher uses less time to give the lecture, students have more time to practice, and videos lectures are provided before class. (3) In a high-flipped classroom (high-FC), the teacher uses the least lecture time, giving students the most practice time, and videos lectures are, again, provided before class.

We investigate and compare learning outcomes of and satisfaction from ERP-system courses in traditional BL, low-FC, and high-FC classrooms through three rounds of quasi-experimental design. The current study has two research purposes. (1) To investigate the differences in learning outcomes between traditional BL and ELT-flipped classrooms, including ERP-system skills and business-process knowledge. (2) To investigate the degree of students’ satisfaction between traditional-BL and ELT-flipped classrooms. We are expecting the study findings could clearly demonstrate the potential significance and practical feasibility of ELT-flipped classrooms. It can also provide the sustainable value of flipped classrooms and e-learning for academic references.

2. Literature Review

We review the ERP system and its value in education in Section 2.1. Section 2.2 discusses experiential-learning theory with ERP. The purpose of the flipped classroom is discussed in Section 2.3. Finally, the research concept is proposed and discussed in Section 2.4.

2.1. ERP Education

The ERP system is a set of information systems that incorporate entire-enterprise processes and activities within an integrated platform to support enterprise operation [2,6]. The ERP system includes all business processes, including finance, accounting, manufacturing, purchasing, inventory, sales and distribution, human resources, etc. [3,5]. It fully supports a company’s cash flow, logistics, and information flow to optimize corporate resources effectively [1]. In 1996, the California State University adopted the SAP R/3 ERP system as its teaching content [35]. Subsequently, many universities have successively offered ERP-system courses [2], which effectively combine practice and theory and are the best way to cultivate students’ ERP-system skills and business-process knowledge [3,5,16,33]. Traditional classroom teaching is the most common and necessary educational method in ERP education [7]. Some teachers, therefore, recommend hands-on tasks that also better support traditional teaching [6,9,16,33].

2.2. Experiential-Learning Theory and ERP

Dewey has advocated for learning by doing [10]; whereas, Kolb has emphasized that people learn through their own experiences and discoveries, which shape their mastery of knowledge, affect their cognitive development [11]. Kolb proposed the experiential learning cycle from Dewey’s “learning by doing” [10], the social psychology of Lewin [36,37], and Piaget’s theory of cognitive development [38]. Experiential learning theory divides the learning process into four repeating cycles. (1) Concrete experience (CE) emphasizes the generation of feelings through specific experiences. (2) Reflective observation (RO) emphasizes that observation and reflection come from experience. (3) Abstract conceptualization (AC) emphasizes learning knowledge, abstractions, and constructs by thinking. (4) Active experiment (AE) emphasizes learning by doing. Dewey, Lewin, and Kolb reported experiential learning had been widely accepted concerning educational practice [14,15]. It is also used in ERP-system education and research [7,8,9,16,33,34].

2.3. Flipped Classroom

The advancement of e-learning technology and educational theory helped flipped classroom to be flourished. The flipped classroom proposes that students watch video lectures at home and reduce or cancel teachers’ lecture time, and students are given more time to complete learning assignments in the classroom through interaction with teachers or classmates. [17,18,19]. The flipped classroom emphasizes the transformation of learning from teacher-oriented to student-oriented [18,19,20]. From a technological perspective, the flipped classroom uses e-learning technology to support learning activities outside the classroom. From the perspective of educational theory, student-centered learning theory is the core value of the flipped classroom [18,19].

The literature explored flipped classrooms could improve learning outcomes [19,20,31,32] through improving academic performance [21,22,39], students’ satisfaction [23,24] and engagement [25,26], and the application of skills [25]. The flipped classrooms also prioritizes learning motivation [27,28], higher self-efficacy [28], the efficient use of class time [40], and more time for practice [41]. Nonetheless, the literature provided evidence that academic performance has not improved greatly using flipped classrooms [26,29,30,42].

The flipped classroom seems only suggest flipping the activities of “in-class” and “out-of-class”. In reality, this is a superficial reading of the phrase [19,20]. The implications of the flipped classroom have far-reaching significance and a major impact on educational concepts. It effectively implements both teachers’ lectures and active student learning at the same time, combining the characteristics of behaviorism and constructivism. Yet, the two ideas were originally incompatible [19].

Flip classrooms are often confused with blended learning. Blended learning generally refers to a learning method that combines traditional face-to-face lectures and distance learning [21,43]. Learners can learn in their own time and according to learning speed [21,43]. The face-to-face teaching and distance-learning teaching context can take several forms and combinations. The definition of blended learning is broad but not specific, and it is not appropriate to regard it as the same as the flipped classroom [19,31], as blended learning can be very traditional and “old school” [21,44].

2.4. ELT-Flipped Classroom and Traditional Blended Learning

Based on the above discussion, we have clarified the traditional blended learning and flipping classrooms in this study.

• Traditional blended learning (traditional BL). It follows a three-step cycle. 1. The teacher lectures in the classroom. 2. Students listen in class and may have little time to practice. 3. Students go home, after class, to do their homework. E-learning video lectures are provided for a review after class. Figure 1 shows the traditional BL learning cycle, sequenced in a clockwise direction.
• ELT-flipped classrooms. ELT highly focuses on learning by doing. It, too, follows a three-step cycle. 1. Before class, students preview video lectures. 2. In class, the teacher gives a short lectures or may even give none at all. 3. In class, students are given more practice time, and the teacher becomes a facilitator. Figure 2 shows the ELT-flipped-classroom learning cycle, sequenced in a counterclockwise direction.

The current study has two research purposes. (1) To investigate the difference in learning outcomes between traditional BL-based and ELT-based-flipped classrooms. (2) To explores the differences in satisfaction between traditional blended-learning and ELT-based-flipped classrooms.

3. Methods

3.1. Research Field and Participants

The research object was a third-year course in the Information Management Department of the University of Science and Technology in Taiwan. The name of the course is “Enterprise Resource Planning System,” which is a 3-credit compulsory course. In the computer classroom, each student had an SAP ERP account. There were 94 students in two classes, 50 and 44 in classes A and B, respectively. Students in both classes had no experience using the SAP ERP system. We divided all of the participants into two groups called experimental and the control groups based on the schedules of the participants.

3.2. Experimental Design and Research Hypothesis

3.2.1. Experimental Design

The research adopts a quasi-experimental design. For research purposes, we designed the following three teaching strategies.

• Traditional blended learning (traditional BL). Traditional BL is a teaching strategy based on teacher lectures and supplemented by e-learning after class. The course steps were as follows.
  • Teacher lectures in class. The teacher lectures with slides and demonstrates the ERP system; lectures account for 75% of class time.
  • Student exercises in class. Students use their remaining time to do exercises after listening to the lecture; practice accounts for 25% of class time.
  • E-learning after class. The teacher provides video lectures for students to review after class, shown in Figure 3.
• Low-flipped classroom (low-FC). Low-FC is a flipped teaching strategy, based on ELT, that gives half of class time for students to practice the ERP system. The course steps were as follows.
  • E-learning before class. Students watch video lectures at home.
  • In-class teacher lecture. The teacher lectures in a more practical way, and the lecture accounts for 50% of the class time.
  • Student exercises in class. Students spend more time doing exercises, the teacher plays an assistant role, and students spend around 50% of class time in practice, as shown in Figure 4.
• High-flipped classroom (high-FC): High-FC is a flipped teaching strategy, based on ELT, that allocates 75% of class time to practicing the ERP system. The course steps were as follows.
  • E-learning before class. Students watch video lectures at home.
  • In-class teacher lecture. The teacher lectures in an essential way, and the lecture accounts for 25% of class time.
  • Student exercises in class. Students spend most of their time doing exercises, the teacher plays an assistant role, and students spend around 75% of class time in practice, as shown in Figure 5.

3.2.2. Research Flow

This study had a preparation stage, and three experimental stages.

• Preliminary stage (4 weeks). The aim was to learn the essential content of the SAP ERP system. In the first four weeks, the teacher traditionally taught SAP ERP. A pre-test was given in the 4th week to confirm differences in the learning backgrounds of the two classes.
• Stage 1 (low-FC vs. traditional BL). The course unit was the procurement process (PUR), which lasted for five weeks. Class A used low-FC, and Class B used traditional BL. The purpose was to investigate the difference in learning outcomes between traditional BL and low-FC.
• Stage 2 (high-FC vs. low-FC): The course unit was materials management (MM), which lasted for five weeks. Class A used high-FC, and Class B used low-FC. In this stage, both classes used flipped teaching strategies. The experiment design aimed to explore the differences in learning outcomes between different levels of flipped classrooms.
• Stage 3 (traditional BL vsvs. high-FC): The course unit was the sales and distribution process (SP), which lasted for five weeks. Class A used traditional BL, and Class B used high-FC. The purpose of the experimental design in this phase is similar to that of the first phase. However, the exchange of paradigm between Class A and Class B improves the accuracy of the experimental results. At the end of each course unit, all students were given a quiz.

Learning satisfaction is closely related to learning outcomes. Researchers have often used satisfaction and learning outcomes as the final evaluation of course learning [43,45,46]. ERP-system skills and business-process knowledge are the main learning content of ERP courses [3,4,5]. Therefore, the quiz for each course unit consisted of two parts. One, regarding SAP ERP-system skills, evaluated students’ technical abilities through computer-based exams. The second, concerning business-process knowledge was assessed with multiple-choice quizzes and assessments of students’ understanding of business processes. In addition, we will also conducted course-unit satisfaction surveys for all students. At the end of the semester, each student had experienced three teaching strategies; a questionnaire survey was used to understand each student’s satisfaction with these three teaching strategies. (Figure 6.)

3.2.3. Research Framework and Hypotheses

The first stage of the experimental design investigates differences in learning outcomes between traditional BL and low-FC. Studies have shown that the learning outcomes of flipped classrooms are better than traditional teaching in many university courses [31,32]. Whether it is entirely flipped or partially flipped, studies have shown that such classrooms can improve learning outcomes [19,20,22,31,32,39]. The literature has also indicated that experiential learning is better than traditional lectures [7,9,16]. Therefore, the current study concludes that low-FC learning outcomes are better than traditional BL, whether in system skills or process knowledge.

Hypothesis 1A (H1A).

Regarding ERP-system skills, low-FC has a better learning outcome than traditional BL.

Hypothesis 1B (H1B).

Regarding business-process knowledge, low-FC has a better learning outcome than traditional BL.

The second stage of the experimental design investigates the difference in learning outcomes between high-FC and low-FC. Both groups belong to flipped learning; the main difference between them is that student practice time in high-FC classes is 75% of class duration, whereas practice time in low-FC classes is 50%.

The second stage of the experimental design investigates the difference in learning outcomes between high-FC and low-FC. Both groups belong to flipped learning; their main difference is that the practice time in high-FC classes is 75% of class time, whereas the practice time in low-FC classes is 50%. As mentioned, Dewey emphasizes learning by doing [10], and experiential learning has been proven to improve learning outcomes [11,12]. More practice time means that students practice more, which in turn means that teachers spend more time in interaction and giving assistance. It is expected that high-FC can elicit better learning outcomes.

Hypothesis 2A (H2A).

Regarding ERP-system skills, high-FC has better learning outcomes than low-FC.

Hypothesis 2B (H2B).

Regarding business-process knowledge, high-FC has better learning outcomes than low-FC.

The third stage of the experiment investigates the difference in learning outcomes between high-FC and traditional BL. The practice time in high-FC classes is 75%, compared with 25% in traditional-BL classes. As discussed in the literature and mentioned above [19,20,22,31,32,39], high-FC can achieve better learning outcomes than traditional BL.

Hypothesis 3A (H3A).

Regarding ERP-system skills, high-FC has better learning outcomes than traditional BL.

Hypothesis 3B (H3B).

Regarding business-process knowledge, high-FC has better learning outcomes than traditional BL.

The above stages investigated the difference in test scores between the experimental and control groups. After students had experienced all three teaching strategies, questionnaires were used to investigate the perceived learning outcomes in each. Therefore, based on the findings of the literature we proposed Hypothesis 4. The learning outcomes of flipped classrooms are better than traditional teaching [19,20,22,31,32,39]. Experiential learning is better than traditional lectures [7,8,16,33,34]. Previous study pointed out flipped teaching can help to reduce the drop-out ratio of the students with lower scores. In addition, partial-flipped teaching achieves the best overall learning results [47].

Hypothesis 4 (H4).

Students have the highest perceived learning outcomes for high-FC, followed by low-FC, and the lowest for traditional BL.

Literature found that the flipped classroom showed higher satisfaction [23,24,43], and more efficient to use the teaching time [40], and practice during the class time [41]. Because high flip has more time and less teaching time than low flip, students got more time to be independent to acquire the knowledge. Students prefer less lecturing time and more practice time. Hypothesis 5 is proposed.

Hypothesis 5 (H5).

Students have the highest perceived learning satisfaction in high-FC, followed by low-FC, and the lowest in traditional BL.

Both high-FC and low-FC provide video lectures before class, whereas traditional BL provides videos afterward. To better understand students’ satisfaction with respect to when video lectures are given, we surveyed students’ satisfaction with both. Hypothesis 6 is proposed according to the findings of learning outcomes, which indicated that flipped classrooms are better than traditional teaching [19,20,22,31,32,39,47], which includes out-of-class activities and in-class activities. Students were therefore expected to favor video lectures before class.

Hypothesis 6 (H6).

Students are more satisfied with preview e-learning than review e-learning.

The research hypotheses are consolidated in the research framework, as shown in Figure 7.

3.3. Instructional Approaches Design

3.3.1. Course Outline and Content

This course ran for 18 weeks, and the course was divided into four stages. (1) The first consisted of an introduction to ERP and essential operation of ERP system for four weeks. (2) The procurement process occupied this stage included the making of master files of materials and suppliers, requisitions, purchases, merchandise receipts, and invoice verification, which lasted for five weeks (3) This stage concerned inventory management, for a period of 4 weeks, and its content included inventory transfer and transfer posting, transaction management, and inventory counting. (4) This stage concerned the sales and distribution processes, for a period of 5 weeks, and its content included customer master files, product master files, inquiries, quotations, sales orders, finished product storage, shipping documents, packing, shipping, and invoices.

3.3.2. Out-of-Class Activities

The video lectures used in traditional BL, low-FC, and high-FC were the same. In traditional BL they were only provided after class, while for low-FC and high-FC they were provided before class. The video content of the course included teacher lectures, ERP-system demonstrations, and videos of process animation. There were more than 50 videos lectures stored on the school’s e-learning platform. Every student was instructed to watch the videos before come to class.

3.3.3. In-Class Activities

In traditional BL, lectures include full explanations explained by the teacher, whereas, in the flipped classroom, the teacher explained only the essentials. After the lecture, students logged-in to the SAP ERP system to practice and completed their assignments in the classroom. Practice time, in traditional BL, is short, and students are allowed to continue their assignment after class to complete it before the next. In the flipped classroom, students are required to practice and complete the entire assignment in class.

3.3.4. In-Class Practical Tasks

The practical task of the ERP system is focusing on learning in the classroom. Therefore, we list the main practical tasks of each stage as follows.

• Preliminary stage (ERP introduction): The practical tasks, at this stage, included emphasizing the basic operations and settings of the system including the settings of the system environment, personal parameter, identifying names of technologies and using these names to execute system transactions, using system modules, opening menus and executing tasks from them, multitasking execution, creating favorites, adding shortcuts to favorites, and so on.
• Stage 1 (procurement process). In the procurement process, students needed to complete the following system tasks in order: (1) material-master maintenance (technical name: MM01/02/03—the content in brackets below is also the technical name); (2) vendor-master maintenance (XK01/02/03); (3) purchase-order maintenance (ME51N)/52N/53N); (4) purchase-order maintenance (ME21N/22N/23N); (5) purchase-order message transmit (ME9F); (6) material receipts based on purchase order (MIGO), single, split, and consolidated material receipts; and (7) invoice verification (MIRO).
• Stage 2 (materials management): In the materials management unit, students need to complete the following tasks: (1) inventory queries (MMBE), all types of inventory, querying inventory according to different conditions; (2) materials movement (MIGO), performing inventory transfers according to different movement types; and (3) execution of physical inventory, consisting of a. creating an inventory file (MI01); b. inputing inventory (MI04); and c. posting inventory differences (MI20).
• Stage 3 (sales and distribution process): The sales- and distribution-process unit included the following tasks: (1) customer-master maintenance (XD01/02/03); (2) trading-goods maintenance (MMH1); (3) inquiry-order maintenance (VA11/15); (4) quotation-order maintenance (VA21/25); (5) sales-order maintenance (VA01/05), both single and combined; (6) delivery-document maintenance (VL01N), both partial and combined; (7) picking goods (LT03); (8) observing the product (VL03N); (9) handling post-goods issues (VL02N); and (10) creating invoices for payment (VF01).

3.4. Measurement—Learning Outcomes, Learning Satisfaction

3.4.1. Learning Outcomes for H1, H2, H3

ERP-system skills and business-process knowledge are the learning goals of ERP courses [3,4,5,6]. These have different characteristics and different evaluation techniques. Skill assessment, regarding the ERP system, was evaluated through a computerized test. Each student used their SAP ERP account to complete the specified system tasks. Teachers gave grades based on what the students had completed. Taking the procurement process as an example, the examination included creating requisitions and purchase orders, printing, receiving goods, displaying inventory, verifying invoices, document flow, etc. (Figure 8.)

Unlike the computerized test of the ERP system, a multiple-choice test was adopted for the assessment of business-process knowledge. Test items referred to the content of the SAP certification. For example, the question “Which department is responsible for creating purchase requisitions?” had options to answer (1) the purchasing department, (2) the business department, (3) the production department, and (4) all departments are possible.

3.4.2. Perceived Learning Outcomes for H4

At the end of the semester, we investigated students’ perceived learning outcomes from the three learning strategies. We designed a questionnaire of perceived learning outcomes, according to the course goals. The questionnaire uses a five-point Likert scale and contained three questions:

• I have effectively learned ERP-system skills in the (PUR/MM/SP) course that used high-FC/low-FC/traditional BL.
• I have effectively learned business-process knowledge in the (PUR/MM/SP) course that used high-FC/low-FC/traditional BL.
• Generally speaking, I have effectively learned knowledge and skills in the (PUR/MM/SP) courses that used high-FC/low-FC/traditional BL.

3.4.3. Perceived Learning Satisfaction for H5

Learning satisfaction is closely related to learning effectiveness [45]. The literature has often used perceived learning satisfaction and outcomes of perceived learning as measures of learning [43,45,46]. According to a previous studies’ ERP course-satisfaction questionnaire [45], for our perceived learning satisfaction questionnaire, which used a five-point Likert scale we adopted three questions:

• I would recommend (high-FC/low-FC/traditional BL) to other students to learn about ERP systems.
• I am satisfied with the quality of the learning experience (high-FC/low-FC/traditional BL).
• I enjoyed (high-FC/low-FC/traditional BL).

3.4.4. E-Learning Satisfaction between Review and Preview for H6

We also studied students’ satisfaction regarding review e-learning and preview e-learning. Our e-learning satisfaction questionnaire used a five-point Likert scale, and for which we chose three items:

• I would recommend the way to provide video lectures (before class/after class).
• I am very satisfied with the learning results of the (before class/after class) video lectures.
• I enjoy the learning experience of watching video lectures (before class/after class).

3.5. Methodology and Data Analysis

The four-stages quasi-experimental process described in this study are used to investigate the effects of the three teaching strategies. (1) Preparation stage: a pre-test was performed using both experimental classes to reveal the differences in socioeconomic contextual; (2) Stage 1 (A vs. B): in this stage, the proposed process investigates the difference between treatment A and treatment B; (3) Stage 2 (B vs. C): Here, we suggest investigating differences between treatment B and treatment C; (4) Stage 3 (C vs. A): in this stage, our process Investigates differences between treatment C and treatment A. This experimental procedure allows the three treatments to be compared with each other, as both classes will have undergone all three treatments. We concede it is an intricate investigational method.

SPSS version 19.0 was used to test the hypotheses of this article. Test methods included independent-sample t tests, one-way repeated-measurement ANOVA, and dependent-sample t tests. The independent-sample t test was used to test the mean differences of dependent variables between the experimental group and the control group. Hypothesis 1, Hypothesis 2, and Hypothesis 3 compared the learning outcomes between two classes in three stages (system skills and process knowledge). As the two classes belong to independent samples, an independent-sample t test was suitable. Hypothesis 4 and Hypothesis 5 respectively investigate the difference between perceived learning outcomes and perceived learning satisfaction among the three teaching strategies. Perceived learning outcomes and satisfaction were both evaluated with questionnaires at the end of the semester. Each student gave answers regarding the three teaching strategies with respect to perceived learning outcomes and satisfaction, which were individual repeated measurements. Therefore, Hypothesis 4 and Hypothesis 5 were suitably analyzed by one-way repeated-measurement ANOVA. Hypothesis 6 investigated the difference in student satisfaction between preview e-learning and review e-learning; since responses regarding satisfaction of these were given separately, these data were considered a dependent sample. Therefore, Hypothesis 6 was suitably analyzed using a paired-sample t test.

4. Results

4.1. Pre-Test for the Two Classes

The pre-test was performed regarding the content of basic system operation and process concepts.

Table 1. Independent sample t test from pre-testing of learning outcomes.

	Mean (SD)		df	t	p
	Class A (n = 50)	Class B (n = 44)
System	75.70 (12.164)	74.66(13.003)	92	0.401	0.689
Process	72.10 (7.149)	71.82 (9.468)	92	0.164	0.870

presents the findings of the pre-tests for the two classes. The results show no significant difference between the two classes in knowledge of ERP-system skills and processes. Both system operation (0.689 > 0.05) and process knowledge (0.870 > 0.05) are significant in pre-test.

4.2. Learning Outcomes between the Control Group and the Experimental Group (H1, H2, H3)

4.2.1. The Difference in Learning Outcomes between Low-FC and Traditional BL (H1)

The first experimental unit was PUR, which investigated learning differences between low-FC (class A) and traditional BL (class B). The results are shown in

Table 2. Independent sample t test of PUR learning outcomes.

	Mean (SD)		df	t	p
	Low Flipped Class A (n = 50)	Traditional Class B (n = 44)
PUR system	78.10 (14.979)	70.91 (16.922)	92	2.186	0.031 *
PUR process	74.60 (8.562)	74.09 (9.230)	92	0.277	0.782

Note. * p < 0.05.

.

The first part of Table 2 shows a significant difference (p = 0.031 < 0.05 *) in the scores on the PUR-system quiz between classes A and B. Class A with low-FC (mean = 78.10, SD = 14.979) is better than class B with traditional BL (mean = 70.91, SD = 16.922). Thus, H1A is supported.

The second part of Table 2 shows no significant difference (p = 0.0782 > 0.05) in the scores on the PUR-process quiz between classes A and B. Class A (mean = 74.60, SD = 8.562) using low-FC is almost the same as class B (mean = 74.09, SD = 9.230) using traditional BL. Thus, H1B is not supported.

4.2.2. Learning Outcomes between High-FC and Low-FC (H2)

The second experimental unit is MM, which investigated the difference in learning outcomes between high-FC (class A) and low-FC (class B). The results of the t tests are shown in

Table 3. Independent-sample t test of MM Learning Outcomes.

	Mean (SD)		df	t	p
	High Flipped Class A (n = 50)	Low Flipped Class B (n = 44)
MM system	84.30 (8.512)	75.23 (11.100)	92	4.476	0.000 **
MM process	75.80 (8.638)	74.20 (8.954)	92	0.878	0.382

Note. ** p < 0.001.

.

The first part of Table 3 shows a significant difference in the MM system-quiz scores between class A and class B (p = 0.000 *). Class A with high-FC (mean = 84.30, SD = 8.512) is better than class B with low-FC (mean = 75.23, SD = 11.110). H2A is supported.

The second part of Table 3 shows no significant difference in the MM process-quiz scores between classes A and B (=0.382 > 0.05). Class A with high-FC (mean = 74.60, SD = 8.562) is almost identical to class B with low-FC (mean = 74.09, SD = 9.230). Thus, H2B is not supported.

4.2.3. Learning Outcomes between Traditional BL and High-FC (H3)

The third experimental unit is the SP which investigates the difference in learning outcomes between traditional BL (Class A) and high-FC (Class B). The results of the t tests are shown in

Table 4. Independent-sample t test of SP learning Outcomes.

	Mean (SD)		df	t	p
	Traditional BL Class A (n = 50)	High Flipped Class B (n = 44)
SP system	83.92 (10.172)	87.75 (7.048)	92	−2.094	0.039 *
SP process	77.40 (7.840)	76.14 (10.830)	92	0.653	0.515

Note. * p < 0.05.

.

The first part of Table 4 reports significant differences in the SP system test scores between classes A and B (p = 0.039 < 0.005 *). Class B with high-FC (mean = 87.75, SD = 7.048) is better than class A with traditional BL (mean = 83.92, SD = 10.172). Thus, H3A is supported.

The second part of Table 4 reports no significant difference in SP process test scores between classes A and B (=0.515 > 0.05). Thus, H3B is not supported.

4.3. Perceived Learning Outcomes (H4)

Regarding the reliability of the Perceived Learning Outcomes questionnaire, Cronbach’s α is 0.964 for traditional BL, 0.977 for low-FC, and 0.962 for high-FC. The threshold t meet Cronbach’s α was > 0.7 (

Table 5. Descriptive statistics of perceived learning outcomes.

	Mean	SD	n	Cronbach’s α
traditional BL	3.3830	0.74609	94	0.964
low-FC	3.9574	0.75765	94	0.977
high-FC	4.3652	0.75106	94	0.962

The p value of Mauchly’s test of sphericity is .811, or greater than 0.05, which meets the assumption of sphericity. A repeated-measures ANOVA determined that the mean of perceived learning outcomes differed significantly across the three classrooms F (2, 186) = 108.675, p = 0.000 (Table 6).

).

A post-hoc pairwise comparison, using the Bonferroni correction, was applied. The traditional BL value is significantly lower than low-FC (3.3830 vs. 3.9574, p = 0.000); traditional BL is significantly lower than high-FC (3.3830 vs. 4.3652, p = 0.000); and low-FC is significantly lower than high-FC (3.9574 vs. 4.3652, p = 0.000).

Table 6. Tests of Within-Subjects Effects.

Source	SS	df	MS	F	Sig.
Classroom	45.783	2	22.892	108.675	0.000
Error	39.180	186	0.211

features three unique comparisons between the means for traditional BL, low-FC, and high-FC. All means met the level of significance (

Table 7. Post-hoc pairwise comparisons of perceived learning outcomes.

Satisfaction		MD
I	J	(I–J)	Std. Error	Sig.
traditional BL	low-FC	−0.574 *	0.069	0.000
traditional BL	high-FC	−0.982 *	0.067	0.000
low-FC	high-FC	−0.408 *	0.065	0.000

Note. * significant difference.

). Students showed the highest perceived learning outcomes for high-FC, followed by low-FC, and traditional BL. Thus, Hypothesis 4 is supported.

4.4. Perceived Learning Satisfaction (H5)

Regarding the reliability of the perceived learning satisfaction questionnaire, Cronbach’s α of 0.990 is recorded for traditional BL, 0.983 for low-FC, and 0.969 for high-FC (MM), which meets the threshold of Cronbach’s α, > 0.7. Validity is also evaluated by other scholars [45] (

Table 8. Descriptive statistics of perceived learning satisfaction.

	Mean	SD	N	Cronbach’s α
traditional BL	3.3546	0.89176	94	0.990
low-FC	3.7872	0.75071	94	0.983
high-FC	4.1986	0.82279	94	0.969

).

The p value of Mauchly’s test of sphericity is 0.121, or greater than 0.05, which meets the assumption of sphericity. A repeated-measures ANOVA determined that the mean of learning satisfaction significantly differs across three classrooms F (2, 186) = 120.819, p = 0.000 (

Table 9. Tests of within-subjects effects.

Source	SS	df	MS	F	Sig.
Classroom	33.485	2	16.742	120.819	0.000
Error	25.775	186	0.139

).

A post-hoc pairwise comparison, using the Bonferroni correction, was applied. As shown in

Table 10. Post-hoc pairwise comparisons of perceived learning satisfaction.

Satisfaction		MD
I	J	(I–J)	Std. Error	Sig.
traditional BL	low-FC	−0.433 *	0.057	0.000
traditional BL	high-FC	−0.844 *	0.057	0.000
low-FC	high-FC	−0.411 *	0.048	0.000

Note. * significant difference.

, the value for traditional BL is significantly lower than low-FC (3.3546 vs. 3.7872, p = 0.000); traditional BL is also significantly lower than high-FC (3.3546 vs. 4.1986, p = 0.000); low-FC is significantly lower than high-FC (3.7872 vs. 4.1986, p = 0.000). Table 10 reports three unique comparisons between the means for traditional BL, low-FC, and high-FC. All values met the level of significance. Students scored the highest perceived learning satisfaction for high-FC, followed by low-FC and traditional BL. Thus, Hypothesis 5 is supported (Table 10).

4.5. E-Learning Satisfaction (H6)

The Cronbach’s α of the review e-learning satisfaction and preview e-learning satisfaction is 0.944 and 0.905, respectively, above the threshold value of 0.7. The paired-sample t test was used to test scores of satisfaction between review e-learning and the preview e-learning. Students’ satisfaction with preview e-learning (mean = 4.1702, SD = 0.71299) is significantly higher than review e-learning (mean = 3.3014, SD = 0.75581), p = 0.000. Thus, Hypothesis 6 is supported. See

Table 11. Paired-sample t tests of e-learning satisfaction.

	Mean (SD)		df	t	p
	Review e-Learning n = 94	Preview e-Learning n = 94
e-learning Satisfaction	3.3014 (0.75581)	4.1702 (0.71299)	93	−16.124	0.000 **

Note. ** p < 0.001.

.

5. Discussion

In this section, we discuss the research results separately.

Table 12. Summary of results.

	Traditional BL	Low-FC	High-FC
H1A: system skills—PUR	lower	higher	-
H1B: process knowledge—PUR	no significant difference	no significant difference	-
H2A: system skills- MM	-	lower	higher
H2B: process knowledge—MM	-	no significant difference	no significant difference
H3A: system skills—SP	lower	-	higher
H3B: process knowledge—SP	no significant difference	-	no significant difference
H4: perceived learning outcomes	lowest	middle	highest
H5: perceived learning satisfaction	lowest	middle	highest
H6: e-learning satisfaction	review e-learning, lower	preview e-learning, higher	preview e-learning, higher

summarizes the results of this study.

5.1. Learning Outcomes of ERP-System Skills (H1A, H2A, H3A)

Hypotheses 1A, 2A, and 3A investigated the learning outcomes for ERP-system skills, and all were supported. Hypothesis 1A shows that low-FC has better learning outcomes than traditional BL. Hypothesis 2A reports that high-FC has better learning outcomes than low-FC. Hypothesis 3A explores that high-FC has better learning outcomes than traditional BL. Our findings are consistent with those of previous studies concerning flipped classrooms [19,20,22,31,39]. The study results resonate with experiential-learning theory [11,12,13], and are supported by previous studies regarding ERP experiential learning [7,9,16,34].

From synthesis of the findings of H1A, H2A, and H3A on learning outcomes for the ERP system, high-FC is better than low-FC, and low-FC is better than traditional BL. This finding is not only important for ERP-system courses, but also for other computer courses. If such courses use flipped classrooms and experiential-learning theories, students’ learning outcomes would be better.

5.2. Learning Outcomes of Business-Process Knowledge (H1B, H2B, H3B)

Hypotheses H1B, H2B, and H3B investigated the differences in the learning outcomes of business-process knowledge. None of these three hypotheses were supported. In other words, traditional BL, low-FC, and high-FC show no significant differences in their learning results for process knowledge. This finding is inconsistent with the proposed hypotheses. This could be due to the following possibilities:

• Lower-level memorization of knowledge can be repeated through video lectures. Bloom’s taxonomy points out that the learning goals of the cognitive-processes dimension, from lowest to highest level, are to remember, understand, apply, evaluate, and create [48,49,50]. Process knowledge, in the current context, is low-level, memorizable knowledge that students can repeatedly watch in videos, before or after class. Therefore, the differences in learning outcomes, here, are not significant. However, ERP-system skills belong to a higher-level application of knowledge. In the flipped classroom, knowledge is simply remembered before class to engage and leave the creative learning for the classroom practice [17,18,19,20]. From this point of view, the results of the current study are still consistent with previous research findings [20,41].
• Lower-level memorizable knowledge is not affected by the experiential-learning flipped classroom. The experience of using an ERP system certainly contributes to the understanding of process knowledge [3,4,5,16,33], but its influence is not as significant as was that for system skills. Perhaps the experience of process should adopt different teaching methods, such as group activities.
• The employed method of evaluating process knowledge was inappropriate. Single-choice questions may not be able to fully assess students’ concepts of the overall process. With the use of a question-and-answer method, or commentary, etc., it may be possible to distinguish differences in learning outcomes.
• Evidence from the literature on academic performance in flipped classrooms is still relatively poor [26,29,30,42]. From this point of view, the results are consistent, but, this may also be a combined result of the above three reasons. This finding leads to other research topica for evaluation.

5.3. Perceived Learning Outcomes between Traditional BL, Low-FC, High-FC (H4)

The results in Section 4.3 show that students have the highest perceived learning outcomes for high-FC, followed by low-FC and traditional BL. Thus, Hypothesis 4 is supported and consistent with the findings of previous studies regarding flipped classrooms [19,20,22,31,32,39]. This finding is also consistent with previous findings regarding experiential learning [7,8,9,16,34]. Thus, Hypothesis 4 is supported and mutually confirmed with Hypothesis 1A, Hypothesis 2A, and Hypothesis 3A.

5.4. Perceived Learning Satisfaction between Traditional BL, Low-FC, High-FC (H5)

The results in Section 4.4 show that students have the highest perceived learning satisfaction in high-FC, followed by low-FC, and the lowest is traditional BL. Hypothesis 5 is supported, and is consistent with previous studies showing that flipped classrooms have higher learning satisfaction [23,24,28,43]. It is also consistent with the results of previous studies that partially flipped classrooms have the highest satisfaction [47]. Learning satisfaction is closely related to learning outcomes, and they are often consistent with each other [43,45,46]. Thus, Hypothesis 5 also reinforces Hypotheses 1, 2, 3, and 4.

5.5. E-Learning Satisfaction (H6)

The results in Section 4.5 show that students are more satisfied with preview e-learning than review e-learning. Thus, Hypothesis 6 is supported. Preview e-learning is the out-of-class activity of flipped classrooms, whereas review e-learning is the out-of-class activity of traditional blended learning. These results are consistent with previous studies of flipped classrooms [31,47], which show the importance of e-learning in flipped classrooms.

6. Conclusions

The current study used a quasi-experimental design to investigate the differences in learning outcomes and satisfaction of traditional BL, low-FC, and high-FC. Several features were found in our results. (1) Our flipped classroom was based on experiential-learning theory. (2) The current study included flipped classrooms and traditional blended learning, both of which use e-learning, which is different from the settings of traditional classrooms used in previous studies. (3) We clearly distinguish high-FC, low-FC, and traditional BL, and through a well-designed experimental process, repeatedly test the research hypotheses.

The study findings have several implications for academics. (1) Experiential-learning theory can be effectively applied in a flipped classroom model, by understanding the principle of Dewey’s “learning by doing.” (2) The flipped classroom can be an effective platform for active-learning theory; it is not only used in experiential-learning theory. (3) E-learning can be organized in the flipped classroom. The flipped classroom has given e-learning a new role and influenced it positively; flipped classrooms and e-learning support sustainable development of both sides of the learning experience. It also has two implications for teaching practice. (1) The flipped classroom and experiential-learning theory are both feasible teaching strategies. The flipped classroom, based on experiential-learning theory, has obtained preliminary verification in an ERP-system course. It can thus be extended to other courses in the future. (2) Fully flipped classrooms have practical difficulties. Partially flipped classrooms are a more feasible and incremental strategy.

There are several limitations to this study that could be addressed in future research. (1) There is still room for improvement in the results of the three t tests of Hypotheses 1, 2, and 3. If A > B and B > C, then A > C seems to be true. However, this still requires careful interpretation. For future research, it is better to have three classes to conduct research, as this will produce the best interpretation effect. (2) Both perceived-learning outcomes and perceived-learning satisfaction should be investigated at the end of the semester. The advantage is that students can compare different teaching methods at the same time. The disadvantage is that the perception of students from recall may be different from their initial perceptions, which may represent error. Our future research plan is to conduct a survey at the end of each course unit. (3) The generalization of the research results has limitations. Not all types of courses are suitable for flipped classrooms or experiential learning. Some courses have inherent limitations, such as history, philosophy, etc. that is unsuitable for this study methodology. The results of this research are more suitable for explaining practical courses. Future research can use other practical courses as the research object to confirm the research results. (4) The University of Science and Technology is oriented towards practical teaching, and these students tend to learn by doing. Whether such results are applicable to students in academic universities is worthy of further research. (5) The effect of flipped classrooms on memorizable knowledge is not clear in this study. The reasons mentioned in Section 5.2 above are worthy to be the subject of future research. (6) This study did not use a fully flipped classroom. This can be an important topic for future research. (7) The demonstrated cooperation between experiential-learning theory and the flipped classroom is only a preliminary result, and further research is still needed.