Introducing ERP Concepts to IT Students Using an Experiential Learning Approach with an Emphasis on Reflection

Abstract

The introduction of enterprise resource planning (ERP) concepts to IT students entails many challenges. Due to the system’s complexity, newcomers need an extensive amount of time to be able to use it independently. Additionally, the learning preferences and characteristics of digital natives differ significantly from previous generations. Therefore, the use of alternative learning approaches is desirable. To achieve the best possible learning outcomes, it is advisable to implement learning approaches that require students’ active participation, for example, an experiential learning approach. A variation is an ERPsim business simulation game, which we used within the ERP systems course. The game is implemented in sequential rounds, whereby each round ends with a review of the collected experiences. The simulation game was used at the beginning of the course to ease the introduction of ERP concepts for IT-related students. This paper is the result of three years of research into the perceived usability of SAP ERP introduced with the business simulation game, combined with the results of a study evaluating students’ opinions, knowledge, and skills. Perceived usability was measured using a System Usability Scale (SUS), while the students’ experiences were gathered using a self-evaluation questionnaire. The study revealed the positive impact of the experiential learning approach that was used. Students evaluated the usability of SAP ERP as OK, and empirical analysis confirmed that the use of the simulation game for introducing the ERP concepts resulted in anticipated knowledge and skills, while increasing the students’ intent for future engagement.

1. Introduction

The highest learning outcome is achieved when students are engaged actively in the learning process [1]. Therefore, the use of learning approaches requiring students’ active participation is recommended. Schön [2] emphasized the importance of the learning-by-doing approach. He described it as an approach allowing us to “think about doing something while doing it” [2]. Based on the Learning Pyramid by National Training Laboratories Bethel, Maine, USA, learning retention when applying hands-on learning is approximately 75% [3]. This is important, especially in large-scale study courses. An example is a course, ERP systems, carried out within the master degree study program on Informatics and Technologies of Communication, aimed at introducing and deepening the knowledge of ERP solutions for IT students.

Enterprise resource planning (ERP) systems are comprehensive solutions integrating a wide range of business modules aiming to offer a uniform view of a business [4]. ERP systems emerged in order to offer effective IT support to organizations covering a variety of business domains [4,5]. Its widespread use began in the 1970s [4], during which time ERP systems were perceived as an extension of manufacturing resource planning (MRPII) [5,6]. MRPII offered IT support for manufacturing, engineering, marketing, finance, and management. However, the full integration of modules was not achieved [6]. On the other hand, ERP systems provided complete business integration, while adding additional modules such as human resource planning, maintenance support, quality, and decision support applications [6,7,8]. ERP systems are presently offered as a commercial product by many vendors [4], e.g., SAP and Oracle, wherein SAP occupies the largest share of the ERP software market [9].

ERP systems integrate different information and business processes covering multiple business areas within an organization [10,11,12], while at the same time supporting configurability in order to meet the needs of users [4]. Due to the size, complexity, and diversity of functions, the introduction of ERP systems to students constitutes a challenging task. Teaching ERP systems is a demanding task for several reasons. In our case, the subject has a limited number of teaching hours when considering the breadth of the domain, especially when considering that the enrolled students do not have any previous knowledge or experiences with ERP solutions or with standard business processes such as planning, procurement, accounting, etc. If we also add the challenges related to teaching digital natives, then adding an innovative learning approach to existing learning approaches could have a positive impact on learning outcomes.

This paper is the result of a three-year study about the adoption of an experiential learning approach used for the introduction of ERP concepts. The aim was to facilitate the first steps in using a complex ERP solution, and to prepare students for further course assignments. We used a business simulation game, ERPsim [13], that is based on SAP ERP. The implemented variation uses a traditional SAP GUI. The game requires the active collaboration of students, and offers an opportunity for users to reflect on their experiences and then use the obtained information in the following rounds of the game.

To analyze the impact of the implemented learning approach, we designed research that would focus on two aspects. Since usability increases user satisfaction and has an important impact on the frequency of use, we researched the perceived usability of the used ERP solution, namely SAP ERP. The second aspect was the investigation of simulation game impact on ERP solution newcomers with regard to user experience and their perception of gained knowledge and skills in various domains. Therefore, the research was based on the following research questions:

RQ1 

What is the perceived usability of SAP ERP, when introduced via a simulation game, from the perspective of an ERP newcomer IT student?

RQ2 

Do students perceive that the use of a simulation game to introduce ERP concepts results in new knowledge and skills?

RQ3 

Do students believe that an introductory simulation game affects their intent in future course engagement?

The research relationships addressed within the study are graphically presented in Figure 1. The study explored the students’ insight into using an experiential learning approach for the introduction of ERP concepts, focusing on their perception of obtained knowledge and skills and impact of the applied approach on their intent for future course engagement. We also looked into the perceived usability of SAP ERP, introduced with the experiential learning approach. The research questions were answered via the results of a three-year study, composed of a usability survey using an SUS questionnaire, and a survey composed of self-assessment statements evaluating students’ acquired knowledge and experiences, as well as their satisfaction and intent for future engagement.

The rest of the paper is organized as follows. Section 2 presents the theoretical background of experiential learning, presenting the meaning of a reflection phase. Further, the characteristics of digital natives are presented, and the concept of simulation games is introduced as an example of the learning-by-doing approach. Section 3 presents the related work and Section 4 presents a study methodology covering the context, participants, data analysis and data collection. Section 5 presents the results of the usability study, and the empirical results of the survey presenting students’ feedback together with a discussion. Finally, the article ends with the limitations, conclusion and future work.

2. Experiential Learning Approaches and Digital Natives

Experiential learning theory defines learning as the “process whereby knowledge is created through the transformation of experience” [14]. Four abilities are needed for effective experiential learning: (1) concrete experiences; (2) reflective observation; (3) abstract conceptualization; and (4) active experimentation [14]. These abilities are joined into two dimensions, the first beginning with concrete experiencing and ending with abstract conceptualization, and the second one going from active experimentation to reflective observation [14]. This represents an idealized learning cycle with learners involved in the recursive process between experiencing, reflecting, thinking and acting, while responding to different learning situations [15]. To achieve the best possible learning outcomes, it is crucial that experiences are upgraded with participants’ reflections [1,16]. Therefore, a reflecting phase represents an important part of experiential learning. Reflective practice was already highlighted by Schön [2], who described it as “a dialogue between thinking and doing, via which the learner becomes more skilled” [17], and interpreted by Osterman and Kottkamp [1] as a “means by which practitioners can develop a greater level of self-awareness about the nature and impact of their performance”. When reflecting, information can be gathered from the analysis of personal experiences, or collected from other research and practice sources [1].

Learning approaches are also an important topic in the domain of higher education. At present, students are effectively digital natives. The name digital natives was proposed by Prensky [18], who described these individuals as “native speakers of the digital language, computers, video games, and the Internet”. Digital natives encountered the digital technology in the early stages of their lives [19], and, as a result, the current generation of students have brought about significant changes in the learning environment [20]. Digital natives can learn at high speeds, since they are used to receiving information quickly [18,20]. They are prone to networking and game-oriented learning [18] and expect a quick response. Gaming has had a particularly significant impact on their educational expectations and opens up the opportunity for collaboration and competition in a goal-oriented environment [21].

The mentioned characteristics of digital natives engendered the need for incorporating new learning approaches into the existing educational process. Various attempts at experiential and reflective learning can be detected in higher education [17,22,23], wherein one of the variations of experiential learning are simulation games [23]. A subset of simulation games are also business simulation games, which started to be developed back in 1998 [24]. One of the representatives is the ERPsim simulation game [13], designed as a learning-by-doing approach for teaching ERP concepts [25]. Students have to run their own business using an SAP ERP solution to develop an understanding about ERP concepts, to experience the integration of modules within ERP solutions and their benefits for the company, and, finally, to develop technical skills for using the selected ERP system [25]. The ERPsim simulation game [13] is available in nine different versions, where each game can be played in a different number of rounds. Each round consists of several steps. The debriefing step encourages students to reflect on gathered experiences and information from each round, culminating in the learning cycle defined by Kolb [14]. In the reflection phase, the participants review and reflect on obtained concrete experiences, which help them to draw a conclusion and act based on the information gathered from the previous round [23].

3. Related Work

Ruhi and Ghatrenabi [26], Ruhi [27] presented a review of ERP programs and teaching practices, among which, ERP simulations can be frequently found. They also proposed a conceptual template for integrating theoretical and applied teaching approaches that can be used within ERP business school curriculum design [26,27]. Another teaching concept was presented by Jaeger et al. [28], which was also designed for business students. Schwade and Schubert [29] presented the use of The ERP Challenge, covering simulation experiences with the ERP system Microsoft Dynamics NAV. The teaching approach was used throughout the semester and the evaluation was done by students from the study programs Information Management, Information Systems, Management and Psychology and Management and Economy. The survey shows encouraging results.

To achieve the best possible learning outcome, it is crucial that the implemented learning approaches are accepted by the participants. Darban et al. [30] examined the construct of a perceived knowledge update within the context of an ERP simulation game. They confirmed that team collaboration effectiveness positively affects individual effort and knowledge update, which leads to an increased intention to learn [30]. Another aspect was researched by Darban and Polites [31] to highlight the students’ learning patterns. Their research model looked into the willingness to learn through perceived radicalness, which has a positive impact on students’ willingness to learn [31]. Chauhan and Jaiswal [32] researched the behavioral intention of using ERP software training. The research partially confirms the UTAUT model in the context of ERP system training [32]. In studies by Darban et al. [30] and Darban and Polites [31], participants were undergraduate students in an introductory level IS course and undergraduate students enrolled in an Introduction to Enterprise System class, respectively. Chauhan and Jaiswal [32] conducted a study in a business school environment. On the other hand, different studies investigate students’ attitudes and obtained abilities while using the ERPsim simulation game. For example, Seethamraju [33] revealed a significant impact of the simulation game on students’ abilities, on the improvement of process orientation, acquisition of integrative skills and the game’s positive contributions to deep learning. The study is based on experiences gathered through the whole semester in a business school, however a small percent of students were IT-oriented students [33]. Cronan et al. [34] measured the participants’ experiences with simulation, learning ERP and their attitude. As the results indicate, business students evaluated the learning experience as positive [34]. In addition, the increase of students’ attitudes towards SAP and ERP knowledge was confirmed by Hwang and Cruthirds [35], who took into account business oriented students. Cronan et al. [36] compared the learning outcomes in the domain of the ERP business simulation game obtained by objective measures with the self-evaluated perception of learning. As the results indicate, there is a significant correlation between self-assessed knowledge and the objective measures Cronan et al. [36], thus justifying the use of self-evaluation techniques when assessing the effectiveness of simulation game learning approaches.

However, despite the variety of available research, studies were not detected evaluating the impact of a business simulation game on the perceived usability of an SAP ERP solution. In addition, studies using a simulation game only as an introduction and a supplement to already available teaching approaches in ERP system courses were not detected.

Monk and Lycett [23] concluded that it is desirable to have the basic understanding of the core business concepts before learning business processes, wherein understanding could be gained and accelerated via hands-on experience with ERP systems. This proved to be particularly significant when teaching ERP systems to IT-related students. Therefore, our research focused on facilitating the first steps of using the complex ERP solution and to efficiently prepare students for further course assignments. In comparison to related work, we used a simulation game approach only as an introduction to the ERP system course, wherein the continuation of the course was done using traditional learning approaches. In addition, our population was IT master’s degree students, with high IT literacy, which was not the case with related studies. In the small portion of studies where IT students were used as a sample, they were undergraduate students and usually mixed with business students. Within the study, we measured both for future use and satisfaction, important factors, perceived usability and obtained knowledge and skills together with students’ intent for future course engagement. If students perceive the usability of an ERP system as high, this can significantly improve the frequency of use of the evaluated system. On the other hand, the obtained knowledge and skills can significantly improve future performance and student satisfaction.

4. Methodology

4.1. Introducing ERP Concepts Using a Simulation Game Approach

To overcome the difficulties of introducing the ERP concepts to newcomers, the simulation game approach was used as an introduction of the ERP systems course carried out for a master’s degree program in an IT-related field. The outline of the ERP system course is presented in Figure 2. The course started with an introductory part, where the experiential learning approach was applied in order to ease the continuation of the course. After the implementation of the simulation game, the course continues with the use of traditional learning approaches, lectures and lab work, and ends with the examination of obtained knowledge.

Since the goal of the ERP systems course is to acquaint IT students with ERP solutions and not to focus on specific business-oriented knowledge, we decided to use the ERPsim Distribution Game as an introductory game. The Distribution Game is the basic ERPsim game, simulating a bottled water distribution company situated in the German market, selling six products in three different regions [37]. Students formed into teams of three or four, wherein the independently divided business roles covered the different business functions and execution of the corresponding transactions using a traditional SAP GUI. As recommended by Léger et al. [37], the business simulation game was played in three successive rounds, where each round simulates 20 days. In the first round, the participants started with the sales process, covering marketing and pricing. The second and third rounds added transactions related to the procurement and planning processes, allowing participants to forecast sales, and perform purchasing transactions together with material requirements planning (MRP). The game outline, together with available transactions within SAP ERP is presented in Figure 3.

To achieve the best possible outcomes, it is advisable to implement the business simulation game according to the recommended steps provided by Léger [25] and Léger et al. [37]. Each round of the Distribution Game consists of several sequential activities, as presented in Figure 4. Brief instructions at the beginning, which allow newcomers to get to know the basics of the upcoming tasks, are followed by concrete experiences, and active experimentation based on abstract conceptualization. While playing the business simulation game, participants gather the information and experiences by simulating concrete scenarios. The simulation game automates some repetitive tasks, allowing participants to focus on the creative part and business decisions, together with the associated functionality within an ERP solution. After 20 simulated days, where one day is represented by one minute, the game is finished and the business results are disclosed to students. The next activity is a debriefing, where the students rethink and reflect upon their decisions based on their experiences and considering the achieved business results. The debriefing is done in two steps, first, within each group, and, secondly, between participating groups, encouraging communication in order to share the obtained thoughts. The last activity in each round is designing the strategy for the next round, based on previously made reflections. With this, the participants are prepared for the next round, starting again at the beginning of the cycle.

The implementation of the experiential learning approach, namely, the business simulation game ERPsim [13], within the EPR system course, started in 2017, and is now running in the third cohort of students. The format and implementation of introductory lectures evolved based on the gathered feedback. In the first year, a three-day introductory workshop was carried out. The students’ collected feedback was presented by Heričko et al. [38], and the impact of the introductory workshop on the students’ perception on SAP usability was presented by Rajšp et al. [39]. In 2018 and 2019, the introductory simulation game was shortened to one day, and, in 2019, the experiential approach was extended throughout the whole course by using additional ERPsim simulation games, namely the Logistics Game and the Manufacturing Game.

4.2. Participants

The paper presents the results of the three-year research. The data were gathered from 53 master’s degree students, wherein participation by years is as follows: in 2017, 18 students; in 2018, 14 students; and, in 2019, 21 students. Details about the participating students are presented in

Table 1. Profile of the study participants.

Characteristic
Number of the participants	53
Age of the participants	21–23
Study program	Informatics and Technologies of Communication
Study program degree	Master’s degree
Year of study	1st
Experiences with ERP solutions	Newcomers
Knowledge about ERP solutions	None

.

All participants were master’s degree students of Informatics and Technologies of Communication, in their first year of the study program. All participants had a high level of IT literacy. Their age varied between 21 and 23 and none of the participants had previous experience with, or knowledge of, ERP solutions.

4.3. Data Collection and Data Analysis

To collect data from the students, we used a survey. Surveys are suitable for gathering self-reported beliefs and opinions from participants [40]. According to the research questions, a survey questionnaire was developed. The development of a research instrument followed the good practices proposed by Neuman [40], and the final questionnaire is presented in Appendix A. The questionnaire consisted of three major parts:

• Part A—Personal data contained relevant information about the participants. The personal data were collected via open-ended questions about their name and surname, age, study program, study program degree and year of study and close-ended questions allowing the participants to self-assess their experiences with ERP solutions and their knowledge, both with the use of a five-point Likert scale.
• Part B—Perceived usability of SAP ERP measured students’ perceived usability of the ERP system. This is done with the use of a SUS questionnaire with a five-point Likert scale.
• Part C—Opinion and feedback about simulation game approach collected students’ personal opinion and self-evaluation regarding different concepts of the experiential learning approach. This part consisted of 13 statements with a five-point Likert scale, measuring the concepts of appropriateness of the approach, obtained knowledge and skills, motivation and competitiveness, debriefing actions, communication with the team and future engagement and commitment.

The data for every cohort were collected immediately after the business simulation game to gather actual and unaffected opinions from the students. The questionnaire was in an electronic form, using an online survey tool. The analysis of gathered data was made based with exported answers using associated analysis techniques. Part A of the questionnaire was analyzed descriptively and for two questions the frequency of answers and an average value was presented. Part B of the questionnaire was analyzed based on SUS scoring rules provided by Brooke [41]. Part C of the questionnaire was analyzed using frequency analysis and average value. The analysis was done for each year separately.

5. Results

5.1. Impact on the Perceived Usability of the ERP System

Perceived usability is important for user satisfaction, and has a significant impact on the frequency of use. Many definitions of usability can be found, among others, the definition by Nielsen [42] and the definition provided in ISO 9241-11:2018 [43]. The ISO 9241-11:2018 [43] defines usability as “the extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use”. On the other hand, Nielsen [42] defined usability with the use of five attributes: learnability, efficiency, memorability, errors, and satisfaction.

Usability can be evaluated using a variety of approaches. An example is the System Usability Scale (SUS) [41], a widely-used questionnaire that is simple to implement and analyse. It was designed to gather the users’ subjective usability evaluation [41]. The SUS questionnaire consists of ten statements, evaluated with a Likert scale, which have to be evaluated immediately after the use of the system, even before the debriefings or discussion [41]. As shown in the surveys [44,45], SUS is the most commonly used standard questionnaire for measuring the usability of the serious game approaches.

Since SAP ERP is a complex solution combining multiple modules and having the user interface loaded with data and input fields, its use for newcomers can be a demanding task. When playing the ERPsim business simulation game, students worked with a real-life traditional SAP GUI interface. The aim of using the simulation game approach for the introduction of ERP concepts was to raise the perceived usability of SAP ERP by changing the students’ focus from the user interface to the game. Therefore, by using the SUS questionnaire, we answered the research question RQ1: “What is the perceived usability of SAP ERP, when introduced via a simulation game, from the perspective of an ERP newcomer IT student?”

The three-year SUS score is presented in Figure 5. The average SUS value in 2017 was 59.17; in 2018, it was 61.61; and, in 2019, it was 61.43. All three cohorts of students, altogether 53 participants, assessed the usability of SAP ERP with SUS scores around 60 out of 100. As presented in the box-plot in Figure 5, the minimal SUS value was 40.00 in 2017, and 52.50 in both 2018 and 2019. On the other hand, the maximum SUS value was 75.00 in 2017 and 2018, and 77.50 in 2019.

Although the SUS score ranging from 0 to 100 offers transparent information, the understanding of the results can be increased if an adjective is added to the score. An example of an adjective rating was proposed by Bangor et al. [46]. Based on the empirical research, they proposed the use of the following adjectives: worst imaginable (mean SUS score, 12.5; standard deviation, 13.1), awful (mean SUS score, 20.3; standard deviation, 11.3), poor (mean SUS score, 35.7; standard deviation, 12.6), OK (mean SUS score, 50.9; standard deviation, 13.8), good (mean SUS score, 71.4; standard deviation, 11.6), excellent (mean SUS score, 85.5; standard deviation, 10.4), and best imaginable (mean SUS score, 90.9; standard deviation, 13.4). Based on the gathered SUS scores, as presented in Figure 5, the students evaluated the usability of the SAP ERP as OK, with scores of 59.17, 61.61 and 61.43, respectively.

Since related works allowing a comparison were not detected, a comparison of results was made with a SAP usability study done in a different environment and domain. Danda [47] carried out usability research on 103 existing SAP ERP users, using the SAP Fiori user interface. As the results show, the average SUS score was 34.47 [47]. The results gathered within our research deviate significantly in a positive way, despite the fact that the interface used was a traditional SAP GUI. Participating students assessed the usability of SAP ERP as OK, while participants in the study by Danda [47] evaluated the usability of SAP ERP as poor. Due to the perceived differences, we can conclude that the introduction of ERP concepts using a simulation game results in a positive impact on the perceived usability of the SAP ERP system.

5.2. An Empirical Survey of Students’ Feedback on Using Simulation Game Approaches

In addition to perceived usability, we also gathered students’ opinions about the selected experiential learning approach, focusing on acquisitions.

Table 2. Empirical results of three year students’ survey.

	Year	Average	5	4	3	2	1
Appropriateness of the approach
simulation game is an appropriate approach for introducing the basic functionalities of SAP ERP.	2017	4.6	61.11%	38.89%	0.00%	0.00%	0.00%
	2018	4.6	64.29%	35.71%	0.00%	0.00%	0.00%
	2019	4.4	47.62%	47.62%	4.76%	0.00%	0.00%
A simulation game is appropriate for learning basic navigation and use of controls in SAP ERP.	2017	4.6	55.56%	44.44%	0.00%	0.00%	0.00%
	2018	4.5	57.14%	35.71%	7.14%	0.00%	0.00%
	2019	4.4	47.62%	42.86%	9.52%	0.00%	0.00%
Obtained knowledge and skills
A simulation game contributes to the development of the technical skills necessary when using SAP ERP.	2017	4.1	27.78%	55.56%	11.11%	5.56%	0.00%
	2018	4.4	42.86%	57.14%	0.00%	0.00%	0.00%
	2019	4.2	23.81%	76.19%	0.00%	0.00%	0.00%
A simulation game demonstrates the need and benefits of the integration of different ERP modules.	2017	4.1	27.78%	55.56%	11.11%	5.56%	0.00%
	2018	4.5	50.00%	50.00%	0.00%	0.00%	0.00%
	2019	4.2	38.10%	42.86%	19.05%	0.00%	0.00%
New knowledge and skills were obtained about using SAP ERP.	2017	4.5	50.00%	50.00%	0.00%	0.00%	0.00%
	2018	4.7	71.43%	28.57%	0.00%	0.00%	0.00%
	2019	4.3	33.33%	66.67%	0.00%	0.00%	0.00%
New knowledge and skills were obtained about collaboration and coordination between roles.	2017	4.3	33.33%	66.67%	0.00%	0.00%	0.00%
	2018	4.7	71.43%	28.57%	0.00%	0.00%	0.00%
	2019	4.4	47.62%	47.62%	4.76%	0.00%	0.00%
Motivation and competitiveness
The goal was not to win, but to obtain new knowledge.	2017	3.4	5.56%	55.56%	16.67%	22.22%	0.00%
	2018	4.1	42.86%	28.57%	21.43%	7.14%	0.00%
	2019	3.7	33.33%	33.33%	14.29%	9.52%	9.52%
The main motivation was getting the best possible result in comparison to other teams.	2017	4.6	55.56%	44.44%	0.00%	0.00%	0.00%
	2018	4.4	57.14%	28.57%	7.14%	7.14%	0.00%
	2019	4.1	42.86%	33.33%	19.05%	4.76%	0.00%
Debriefing actions
The team followed a certain strategy.	2017	3.2	5.56%	33.33%	44.44%	11.11%	5.56%
	2018	4.0	28.57%	50.00%	14.29%	7.14%	0.00%
	2019	3.7	14.29%	47.62%	28.57%	9.52%	0.00%
The teams shared their experiences after each round.	2017	3.4	5.56%	50.00%	27.78%	16.67%	0.00%
	2018	3.0	14.29%	14.29%	42.86%	14.29%	14.29%
	2019	3.9	28.57%	47.62%	14.29%	4.76%	4.76%
The exchanged information after the rounds was useful, and contributed to a better result in the next round.	2017	3.6	16.67%	50.00%	16.67%	11.11%	5.56%
	2018	3.1	21.43%	7.14%	35.71%	28.57%	7.14%
	2019	4.0	33.33%	38.10%	19.05%	9.52%	0.00%
Communication within the team
Communication between team members was good.	2017	4.3	38.89%	55.56%	5.56%	0.00%	0.00%
	2018	4.9	92.86%	7.14%	0.00%	0.00%	0.00%
	2019	4.7	71.43%	28.57%	0.00%	0.00%	0.00%
Future engagement and commitment
The attendance of lectures will increase due to the simulation game.	2017	3.2	16.67%	22.22%	27.78%	27.78%	5.56%
	2018	3.9	28.57%	42.86%	21.43%	7.14%	0.00%
	2019	4.0	28.57%	38.10%	33.33%	0.00%	0.00%

summarizes the results of a three-year survey, providing the answer to research questions RQ2 ( “Do students perceive that the use of a simulation game to introduce ERP concepts results in new knowledge and skills?”) and RQ3 (“Do students believe that an introductory simulation game affect their intent in future course engagement?”).

The questionnaire gathered students’ opinions and experiences after playing the business simulation game. Students evaluated the given statements using a five-point Likert scale. The answer 5 stands for strongly agree, 4 for agree, 3 for neither agree nor disagree, 2 for disagree and 1 for strongly disagree. The empirical analysis of the results is presented in Table 2. The results are displayed by year, showing an average value for each year, followed by a detailed analysis presenting the frequency of each response in percentages.

As the results indicate, students have a positive opinion about the use of a simulation game approach for introducing the basic functionalities of SAP ERP, and also for learning the basic navigation and use of controls in SAP ERP. All three cohorts evaluated the mentioned aspect with average values from 4.4 to 4.6 on a scale of 1 to 5. The majority of students confirmed that the use of a simulation game contributed to the development of the technical skills necessary for using SAP ERP. In 2018 and 2019, all of the students agreed or strongly agreed with the statement, while, in 2017, the percentage of agreement was 83.34%.

When asked about the newly obtained knowledge and skills about SAP ERP use and collaboration and coordination between different roles, students in 2017 evaluated the statements with 4.5 and 4.3 out of 5, while in 2018 they evaluated both statements with an average of 4.7, and in 2019 with an average of 4.3 and 4.4. The statements looked into students’ perceptions of obtained knowledge, which contributed to answering the research question RQ2. The results are summarized and presented visually in Figure 6. All three groups, without disagreeing, confirmed that the use of a business simulation game resulted in anticipated knowledge and skills connected to collaboration and coordination between roles and for the use of SAP ERP. With regard to the findings provided by Cronan et al. [36] that there is a significant correlation between self-assessed knowledge and objective measures, the gathered results, due to the self-assessment type of questionnaires, seem to provide credible and valid results.

Two of the statements in the survey evaluated the students’ competitiveness and motivation to win. As presented in Section 2, digital natives respond well to game-oriented learning [18], wherein competition in a goal-oriented environment suits their characteristics [21]. The results are presented in Table 2. The students from 2017, with an average value of 4.1, evaluated that their goal was to obtain new knowledge and not to win. In 2017 and 2019, the average values were lower, i.e., 3.4 and 3.7, reflecting the competitiveness of participating students. This was confirmed additionally within the statement measuring the motivation of getting the best possible results in comparison to other teams. The statement was assessed with an average value of 4.6, 4.4 and 4.1.

The debriefing phase in each round, aimed at reflecting the gathered experiences and, based on this, reforming the strategies used in a business simulation game. As the results demonstrated, in 2017, only 38.89% of participants followed a certain strategy during the game. In contrast, in 2018 and 2019, the percentage increased to 78.57% and 61.91%. In 2017, 55.56% of students thought that the teams were willing to share gathered experiences with others, while the number in 2018 is significantly lower, namely 28.58%. However, the average result in 2019 was even higher than in 2017, 76.19%. The mentioned results coincide with the results connected to the usefulness of the exchanged information. The result was the lowest in 2018, with an average value of 3.1, and the highest in 2019, with an average value of 4.0.

Regarding communication within the teams, the results suggest that the communication was good, with average values of 4.3, 4.9 and 4.7. This could be expanded upon with the findings presented by Hwang [48] that teamwork, where one of the dimensions is also interaction with teammates, has an impact on team satisfaction as well as on team effectiveness. The findings are based on the survey results, in cases where students were using the ERPsim business simulation game. Hwang [48] concluded that the used business simulation game is also suitable for teaching and learning teamwork.

The motivation for the use of new learning approaches could also be gathered from the statement where students evaluated their future intent for attending lectures due to the use of the simulation game. The statement contributed to answering the research question RQ3. The results are visualized in Figure 6. The average number increased through the years—3.2 in 2017, 3.9 in 2018, and 4.0 in 2019—showing the positive affect of using the simulation game in the ERP systems course. Since it is possible to detect a significant drop of attendance in lectures in recent years, the results show that the experiential learning approaches incorporated into traditional learning strategies could also increase the students’ level of commitment and engagement.

5.3. Discussion

As the results indicate, the introduction of ERP concepts using a business simulation game was very well accepted by the students. They confirmed the appropriateness of the used experiential learning approach and confirmed that the obtained knowledge and skills were needed for the efficient use of ERP systems. More specifically, they confirmed that the gathered technical skills and knowledge needed for using the SAP ERP, and also the skills about collaboration and coordination between different roles in the organizations and benefits of the integration of different ERP modules. The results also provide insight into the other aspects that are part of experiential learning approaches, such as motivation, competitiveness, and actions within the debriefing phase together with communication between and within teams, whereby the results confirm that digital natives respond well to competitive environments. Another aspect that has a significant impact on satisfaction and frequency of use is the perceived usability of the software that was used. In the study, we calculated the SUS score based on the students’ answers. The score displays the perceived usability of SAP ERP as seen from the perspective of an IT student and ERP newcomer. If the perceived usability would have been low, this would have a negative impact on frequency and satisfaction of use. However, based on the results, the perceived usability is OK, since, with the use of the business simulation game, the focus is transferred from the complex user interface to the simulation game. The student thereby unconsciously conquers the basic ERP concepts.

The main expectation of using a simulation game approach as an introduction to the course was to increase the attendance of the lectures. Trends show low motivation among enrolled students, a high dropout rate and low engagement. With the use of an approach that requires an active amount of student collaboration, the potential to resolve the above-mentioned challenges arises. Based on the results, the intent to attend lectures is higher every year. The students’ positive attitude is also confirmed with the results of the official university students’ survey. The results are presented in

Table 3. Results of official students’ survey.

	Before	After
Overall course ratting	1.61	1.88
Cooperation with students	1.73	1.93
Learning outcome	1.55	1.87

.

The scale of the survey was a five-point scale, ranging from −2 to 2, where −2 stands for very bad and 2 stands for very good. After the implementation of the business simulation game approach, the overall ERP systems course rating increased from 1.61 to 1.88 on a scale from −2 to 2. Additionally, 93.3% of students evaluated the cooperation of teachers with students as a 2, and 86.7% graded the learning outcome as a 2, again on the five-point scale from −2 to 2.

To ensure student engagement in study courses it is crucial that study materials and learning approaches are adapted to today’s generations. Since the fourth goal of the United Nations Sustainable Development Goals is quality education [49], new and adjusted learning approaches are required when teaching digital natives. Therefore, the implemented study contributes to the theory and practice of experiential learning in multiple areas. First, with the applied course outline, which was not used in any of the related studies. The outline predicts the use of experiential learning approach as the introduction of ERP concepts, before continuing with traditional learning approaches. This could increase the students’ motivation and at the same time enough knowledge and skills are obtained to overcome the initial challenges connected to complex study materials. Therefore, the presented approach could be included in any existing study course without completely replacing the existing teaching methods and changing the study materials. Finally, an important contribution is also the measured usability of SAP ERP system as seen from the ERP newcomer IT students’ perspective.

6. Limitations

The results of the implemented study can be affected by different factors. The limitations and potential threats to validity are subsequently presented. The results are biased due to the participants’ subjectivity, since the questionnaires were based on self-assessment statements. The study presents a student’s insights into the used experiential learning approach. The study was made in an academic environment, wherein the approach was used for the classroom introduction of ERP systems to ERP newcomers. The study was carried out in an IT-related study program; consequently, the perceived usability of SAP ERP may differ if evaluated by students with different backgrounds and previous knowledge. The study was done using an SAP ERP solution and the traditional SAP GUI, as the used business simulation game runs on SAP ERP. Therefore, the results cannot be generalized to all ERP solutions.

7. Conclusions

This paper summarizes the results of three years of research aimed at evaluating the appropriateness of the experiential learning approach to ease the use of a complex ERP system for newcomers. The learning approach was implemented in an academic environment with master’s degree students. A variation of the experiential learning approach, the ERPsim business simulation game, was incorporated into the ERP systems course in the master’s degree of the IT-related study program. The simulation game was used at the beginning of the course for the introduction of basic ERP concepts. The game followed an outline recommending the implementation in sequential rounds, focusing on the reflection of gathered experiences and use of the collected information for planning future business strategies. At the end, a survey was performed, measuring the perceived usability of SAP ERP and students’ self-assessment of obtained knowledge, skills, and their intent for future engagement and collaboration within the ERP systems course. The presented results show that this experiential learning approach was positively accepted by the students and provided the expected knowledge and skills needed for the easier continuation of future assignments. At the same time, students shifted their focus from complex study content to a simulation game and unconsciously grasped the fundamental use of a complex system user interface. As the results show, all three groups perceived the usability of SAP ERP as OK, with scores of 59.17, 61.61 and 61.43. Furthermore, the results of the survey indicate that, with the use of a simulation game approach, students acquired new knowledge and skills in the domain of using SAP ERP, and also in the domain of communication and collaboration. The answers show positive feedback towards a reflection phase and its usefulness and, finally, confirmed the increased intent for future attendance of course lectures.

Since the result of the implemented study spoke in favor of adopting the experiential learning approaches as an addition to traditional learning practices, research in this area will be continued. As for future work, we will investigate the acceptance and appropriateness of additional ERPsim games. With this, the suitability of other ERPsim games could also be assessed. In addition, a test allowing an objective assessment of obtained knowledge and skills, focusing on detailed domains, would need to be prepared. With this, self-assessed knowledge could be compared to knowledge measured with an objective method.