Next Article in Journal
Predicting Student Performance Using Clickstream Data and Machine Learning
Next Article in Special Issue
Gamification in a News English Course
Previous Article in Journal
Correspondence between Professional Learning Expectations and Learning Opportunities in Financial Management Textbooks
Previous Article in Special Issue
Room2Educ8: A Framework for Creating Educational Escape Rooms Based on Design Thinking Principles
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Education Sciences
Volume 13
Issue 1
10.3390/educsci13010016
education-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Factors Influencing the Effectiveness of Serious Gaming in the Field of Vocational Orientation

by
Christopher Keller
1,*,
Anna K. Döring
2 and
Elena Makarova
1
1
Institute for Educational Sciences, University of Basel, 4132 Muttenz, Switzerland
2
Centre for Psychological Sciences, University of Westminster, London W1W 6UW, UK
*
Author to whom correspondence should be addressed.
Educ. Sci. 2023, 13(1), 16; https://doi.org/10.3390/educsci13010016
Submission received: 5 December 2022 / Revised: 20 December 2022 / Accepted: 20 December 2022 / Published: 23 December 2022
(This article belongs to the Special Issue Game-Based Learning and Gamification for Education)
Browse Figures
Review Reports Versions Notes

Abstract

:
This study investigates the effectiveness of the serious game like2be, which has been developed to support the individual career orientation process of adolescents by broadening their occupational horizon. In this paper, we present results from an intervention study with n = 809 adolescents in Swiss schools at the lower secondary education level. To analyze the extent to which cognitive, affective, and motivational factors are stimulated and what influence they have on expanding knowledge about occupations (measured learning outcome), we applied confirmatory factor analysis, multiple linear regression, and a structural equation model. The results indicate that the stimulation of cognitive processes through serious gaming has a statistically significant impact on learning outcome, although such factors as enjoyment, flow experience, or self-perceived benefits in playing like2be did not significantly impact gain in knowledge about occupations.

1. Introduction

Technological development is advancing rapidly with considerable impact on education worldwide. One such advance in the educational field is learning through the use of serious games. Their implementation in the classroom not only enriches conventional teaching methods but has also become a popular research topic [1]. Serious games contain major pedagogical potential, as they have been explicitly developed for learning purposes [2,3]. They have been used for a number of different educational purposes [4,5], including in the field of vocational orientation [6,7,8].
Vocational orientation acquires special importance towards the end of compulsory schooling as adolescents need to decide which career they want to pursue. Career choice is a major biographical event for young people as it strongly relates to individual development and life experience [9]. Furthermore, it is embedded in a personal and multi-layered career orientation process that begins in early childhood [10]. In a career choice process, adolescents strive for a professional career that fits particularly well with their career-related personal characteristics. In the context of career research, the level of fit between these characteristics and those of the professional environment (occupational fit) indicates the probability of professional satisfaction, commitment and career stability [11]. However, occupational fit is a complex construct. It emerges and is transformed through interaction with the environment [12] and makes career orientation a lifelong process.
From early childhood, young individuals begin to develop their own personality, self-concept [13], and role in their social environment [14]. In the process, different interests, skills, competencies, and talents emerge, as well as values, strengths, and limitations. These are important indicators for the identification of career-related personality traits and for the formation of a career self-concept with which to find a fitting entry into the occupational world [13,14]. However, children also learn about different professions at an early age. They recognize occupational activities and functions and learn to distinguish between them. Moreover, in interaction with their social–cultural environment, they observe not only who is typically employed in certain professions, but also what prestige is ascribed to certain professions. In consequence, the children develop their first occupational stereotypes (e.g., occupational gender roles), which influence their future career orientation [15].
Because career choice is closely related to individual development and can be influenced by a variety of factors, it is important that young learners engage in career exploration. Research in the field of vocational education and training (VET) shows that their transition into the professional world is particularly successful when they are intensively engaged in in their own career exploration process [16,17,18]. For this reason, the serious game like2be was developed so that adolescents can learn about professions in a playful way and thus broaden their own career horizons [2].
Since serious gaming offers an innovative alternative to conventional learning methods, empirical research in this research field has been intensified in recent years [1]. To date, however, there is a lack of empirical evidence on effectiveness of serious games for career orientation purposes. To overcome this research gap, we conducted a quasi-experimental intervention study in order to discover whether like2be effectively supports learning in the context of career choice. The data indicated that the integration of the serious game like2be into school lessons can effectively support adolescents in acquiring knowledge about professions, thereby broadening their career horizons [19].
In the present study, we applied a confirmatory factor analysis (CFA) and calculated a structural equation model (SEM) to provide a differentiated answer to the following research question: To what extent do cognitive, affective, and motivational factors influence learning with the serious game like2be?

1.1. Serious Games

The term serious game covers a wide range of attributes. First, a serious game is basically a game, and represents a series of voluntary and enjoyable activities in which players are involved. Furthermore, it is governed by constraints or rules, and it involves certain goals and possibilities for achieving these goals through moves or actions [20,21]. Second, serious games are usually developed by experts explicitly for a specific target group and their abilities and needs, and since concrete learning objectives as well as media-didactic models are taken into account during development, they lead to an entertaining mental contest and are able to stimulate learning processes effectively [3].
Although serious games do provide entertainment, their main purpose is learning [22,23,24]. As such, they are primarily used for educational purposes [22], but are occasionally used with the aim of supporting attitude and behavior change, e.g., to discourage smoking or promote recycling [25,26]. The term Serious Game also includes the sub-category Digital Game-based Learning (DGBL), which refers to an innovative approach to learning skill acquisition and training through computer games (e.g., serious computer games) and therefore has high educational value and potential [26,27]. However, it should be noted that the two terms (Serious Computer Gaming, DGBL) are sometimes used synonymously [28]. In this paper, the terms Serious Game and Serious Gaming are used to refer exclusively to computer-based games.

1.2. Serious Gaming and Subject Learning

Serious Gaming has become a popular research topic. In particular, the past few years have seen research into whether serious games benefit learning in specific subject areas. Although studies have demonstrated that serious games can be used effectively for various “exotic” educational purposes, such as to reduce school phobia [5], to raise students’ awareness of Internet dangers [29], for vocational training purposes [30,31,32,33], particularly in commercially oriented businesses [34], or for addiction and disease prevention [35,36], serious gaming has principally been analyzed for its effectiveness in the context of subject-related learning.
For example, Byun and Joung [37] showed that serious games significantly benefited the learning of mathematics among students in grades K-12, although the effect was considered small and the data analysis implied that there may be other options for learning mathematics more effectively than with serious gaming. Likewise, Tokac and colleagues [38] mentioned that serious gaming leads to small but significantly higher learning gains for students in PreK-12th-grade compared to traditional instructional methods. Talan and colleagues [39] similarly described the effect of serious gaming on mathematics learning as moderate, but, unlike in previous studies, they included outcomes from preschool to university in the analysis. Further, in a currently published review, Hussein and colleagues [40] also reported that serious gaming positively impacted mathematics learning in K-12 levels. Although the studies reviewed focus on a wide range of mathematics-related topics, most studies were limited to the effect of serious gaming on learning arithmetic operations.
The effectiveness of serious games for language learning has also been addressed in various studies. Chiu and colleagues [41] found that serious gaming leads to a medium positive effect size with regard to foreign language learning. They noted that drill and practice games result in a small positive learning effect, while meaningful and engaging games result in a large positive effect size. Talan and colleagues [39] also showed a large effect of serious gaming on language learning. They therefore suggest that second language learning is a promising area in which serious gaming could be more effective than traditional media for students from preschool to university. Similarly, Chen and colleagues [42] mentioned large effects of serious gaming on language learning, especially on vocabulary acquisition. In this regard, a gaming approach appears to be superior to conventional learning methods. Similarly to a previous study, their results suggest that the effects of serious gaming on language learning depend on the characteristics of the game design. In particular, adventure games seem to be more challenging for players and thus prove more effective for learning outcomes.
As in learning math and language, research has shown that serious games have a positive impact on science learning achievements. For example, Tsai and Tsai [43] compared science learning using conventional activities with learning supported by playing serious science games. They report that compared to conventional instruction in science classrooms, students learn significantly better with serious games in science class. Also, Riopel and colleagues [44] showed that serious gaming is an effective alternative for science learning. Especially in terms of knowledge acquisition, knowledge learning and knowledge retention, was serious gaming shown to outperform learning with more conventional methods under conditions of similar time investment or engagement. Likewise, Hu and colleagues [45] describe science learning through serious gaming as particularly effective. They also note that serious gaming had a positive effect on cognition, emotion, motivation, and retention and thus led to learning effects. Finally, Lei and colleagues [46] also examined the effects of learning with serious games compared with traditional teaching methods on the academic achievement of students from elementary school through university. They found that students learned substantially more with the use of serious games than under traditional instruction. As possible moderation effects, the authors consider an increase in student autonomy or motivation.
Overall, the state of research shows that serious gaming is used for different purposes in the school context and supports learning, especially subject learning (e.g., mathematics, language or science). Furthermore, it shows that serious gaming can be used to create conditions conducive to learning.

1.3. Serious Gaming and Factors That Promote Learning

The cultural historian Johan Huizinga noted that the human species was once called Homo Sapiens. When man himself realized that he was probably not as reasonable as assumed, Homo Faber was created, man the maker. However, since humans have always also been playful beings, the human species, in addition to Homo Sapiens and Homo Faber, can also justifiably be labeled Homo Ludens, man the player [47]. Archaeological research has revealed that mankind engaged in play thousands of years ago. Mostly natural materials such as stones, fruit, grains, wood, and later bones were used for different forms of games. Excavations also confirm that the ball, the doll, and toy animals are among the oldest objects of play. Furthermore, ancient games such as nine men’s morris or chess are still played with great fascination today [48]. Moreover, it is emphasized by historians, e.g., Retter [49], that as early as approx. 5000 B.C. the peoples of present-day China, India, Persia, Egypt and Greece began to think about playful activities and their importance in education. Consequently, it can be assumed that humans have always had a marked tendency to play games, which gives games and play a considerable potential.
However, the potential of serious games goes far beyond the historical perspective. On the one hand, serious games fit in with trends of the current age, and on the other, they are exclusively available on devices (e.g., smart devices) that have great influence on people’s everyday lives. Concerning the professional development of serious games, many studies have reported that serious gaming has a positive effect on specific learning of subject knowledge among learners of different ages. Other studies have focused less on the learning outcome (i.e., subject related knowledge), but instead on factors that can promote effective learning [4], such as an increase in cognitive performance or the stimulation of flow experience, of emotions (e.g., enjoyment), as well as of self-perceived motivating aspects (e.g., learning benefits through serious gaming).

1.3.1. Stimulation of Cognitive Processes with Serious Gaming

In a meta-analysis on serious gaming and its influence on cognitive processes and motivation, Wouters and colleagues [50] found that serious games are more effective in learning and retention, but no more motivating than conventional instruction methods. However Mao and colleagues [51] mention that serious gaming is particularly effective because it often focuses on problem-based learning, where players have to try different strategies to solve problems in order to progress within the game. Additionally, Jong and colleagues [52] found that serious gaming has a positive learning impact when players perceive the game as a challenging task that also offers them the prospect of winning. Furthermore, Sailer and Homner [53] find that gamified learning methods have significant but small overall effects on cognitive, motivational, and behavioral outcomes. Although they point out that gamified learning methods and serious gaming share the same focus on learning value beyond entertainment, but they differ in nature. They also add that collaborative and competitive aspects in learning games are of great importance for cognitive stimulation. Moreover, serious games have been effectively used to increase players’ cognitive performance level in terms of fostering attention capacity [54], enhancing short-term and visual memory performance [55], and reducing the fear of learning failure, thus having a positive effect on learning motivation [56].
All studies showed that serious gaming stimulates cognitive processes and that this has a positive effect on learning. Here, the focus is on increasing motivation. A similar relationship exists between enjoyment and effective learning.

1.3.2. Enjoyment through Serious Gaming

Wouters and colleagues [50] mention that it is generally assumed that serious games have a similar motivational appeal, such as high entertainment value, much as commercial computer games have. Nevertheless, they add that serious games cannot compete with commercial computer games in terms of gameplay, game content and game design and therefore have a less motivating effect. This is especially evident in the sandbox game MineCraft. Although it is not a serious game by definition, considerable effort has been put into the development of an Educational Edition in recent years. This edition was developed by educators for educators and includes innumerable possibilities to use MineCraft in a school context in a way that enhances learning processes. Owing to the Educational Edition, MineCraft can be understood as a kind of serious gaming. In various studies, MineCraft is said to have great educational potential, which is related to its high level of enjoyment [57,58,59]. One indicator for this is its open, creative gameplay which encourages exploration and learning—even requires it [60].
Moreover, self-confidence and self-efficacy are increased through serious gaming [56,61], which is strongly related to the enjoyment of playing serious games, and which contributes to motivational or behavioral learning outcomes [54,61]. Although Breien and Wasson [62] consider the effect of serious games on learning outcomes to be positive in their review, a narrative or story included in the gameplay was found to be a key aspect of the effectiveness of serious gaming. In particular, they identified four narrative or story-related possibilities that increase the positive impact of serious games on enjoyment, engagement, and, consequently, learning: a virtual, quest-based game landscape that can be explored in-game, changeable objects used to overcome challenges and achieve goals, and game-relevant avatars whose own story makes an exciting contribution to the game story, or the integration of real, significant events in human history into the game story.
According to Iten and Petko [29], however, although some scientific evidence suggests that serious gaming may enhance motivation and positive emotions (especially enjoyment), which are strong factors in positive learning outcomes, connections between fun and learning have not yet been fully analyzed by empirical research. They found no clear connection between fun and learning success in their intervention study evaluating the effectiveness of a serious game in Swiss schools. Yet their data showed that greater enjoyment of serious gaming led to greater interest in the learning content, and so they concluded that serious gaming is an effective alternative for introducing a new topic and increasing motivation during the learning process [29].
Thus, studies have shown that serious games can be fun and entertaining and that they consequently promote learning. Nevertheless, it is assumed that they must provide players with a very high level of enjoyment in order to be effective for learning. Another factor related to effective learning is the experience of flow.

1.3.3. Stimulation of Flow Experience with Serious Gaming

Serious gaming can foster intellectual competition in an entertaining way [22,23,24], and because it is an act of doing (i.e., learning by doing) it contributes to a flow experience. If certain factors are present during the game (such as comprehensible game goals, rules and gameplay, feasible challenges, player concentration, or feedback or assistance), players can fall into a flow state [63]. According to Csíkszentmihályi [64], this state is a situation of complete absorption or engagement in an activity and is of great importance to learning processes. If learners additionally experience a particularly high level of enjoyment while playing a serious game, they can fall into a GameFlow state [65]. Fu and colleagues [63] assume that players in a GameFlow state increase their motivation and engagement, which positively stimulates their learning process. Finally, Wronowski and colleagues [66] reported that students who used a serious game (Deadly Distribution) for the purpose of learning statistics were highly absorbed during gameplay and showed higher levels of engagement and ultimately interest in the subject of statistics than those who used conventional methods.
For those who are completely immersed in an activity, totally focused and exclusively engaged, we talk about the mental sensation of flow. In the context of learning, this creates an environment conducive to learning. Research has shown that serious games can lead to such flow experiences. In addition to stimulating the flow experience, serious games can have an impact on self-perceived benefit through the activity.

1.3.4. Self-Perceived Benefit through Serious Gaming

Although serious games effectively promote learning, the use of supplemental materials appears to further increase learning outcome. Wouters and colleagues [50] noted that serious games were most effective for learning processes when supplemented with other teaching materials rather than when used as the sole teaching method. In a learning process, serious games in combination with specific didactic support (e.g., reflection, modeling, collaboration, modality, feedback, or personalization) led to well-structured prior knowledge that helped learners to build on and continue to learn successfully [50,67]. Similarly, Chen and Law [68] mentioned the benefit of additional support. They analyzed the effectiveness of in-game scaffolding in the serious gaming process and concluded that with additional scaffolding, the positive effect of serious gaming on student motivation and learning performance was significantly enhanced. Moreover, for Mao and colleagues [51], additional support also represents an important aspect of effective learning with serious games. If players receive beneficial feedback while gaming or afterwards, it helps them to better understand and reflect on information use as well as on the decisions made in the serious gaming process, which ultimately supports learning. In order to effectively support learning processes and make learning effects visible, learning options such as serious games must be methodologically and didactically embedded in the school context [69].
In other words, serious games can support learning processes, but purposeful application and the use of additional learning materials can strengthen the learning effect. It appears to be important that additional deepening of the subject matter of the serious games consolidates learning on the one hand, and on the other, the players come to recognize that they can learn successfully through serious gaming.

1.4. like2be—A Serious Game for Vocational Orientation

In school lessons on “Vocational Orientation”, adolescents at the Swiss secondary level are specifically stimulated for an intensive and individual vocational exploration phase, the aim of which is to help them succeed in making the transition to the world of work [70]. An innovative possibility for exploration in the context of career choice is the serious game like2be. [6,71] Like2be was developed as part of a research project supported by the Swiss National Science Foundation (SNSF) with the involvement of various experts. It is a web-based online game and can be played for free in German, French and Italian (www.like2be.ch). In addition, this serious game was developed specifically for adolescents in their career orientation process who will have to make an initial career choice decision by the end of compulsory schooling [2].
With like2be, young people can expand their career choice horizons in a playful way. It is a simple point-and-click game without narrative. As players, they take on the role of a personnel agent who must place applicants in suitable jobs or training positions based on their application folders and CVs within a specified time (see Figure 1). A suitable placement is followed by promotion at the end of the game round and the level of difficulty increases (i.e., more vacant jobs to choose from). In the case of unsuitable placements, the player is threatened with dismissal from the virtual job agency [2].
Like2be includes different jobs or training positions, a variety of cartoon-style game characters (avatars), and various personal profiles (application folders and CVs). The jobs or training positions integrated into like2be are taken from the official Swiss information portal for career, study, and career guidance (www.berufsberatung.ch). The serious game like2be does not contain a specific roster. Instead, a randomization mechanism ensures that all application folders and CVs occurring in the game are randomly assigned to a game character (avatar) each time the game is started. Consequently, players can thereby only play successfully if they compare skills, abilities, and individual wishes of applicants with the job jobs or training positions offer [2].

2. Study Design & Methodology

In 2021, we conducted a quasi-experimental intervention study with adolescents at the secondary school level to analyze the extent to which like2be can broaden young adults’ career choice horizons. The total sample included n = 809 adolescents from German-speaking Switzerland. Of these, 49.4% were female, 48.2% were male, 2% assigned themselves to another gender. Their average age was 13.77 years (SD = 0.82). After incomplete data were excluded, a final sample including 532 adolescents remained. Of these, n = 415 were in the intervention groups and played like2be in the classroom; 48.7% were female, 50.1% were male, and 1.2% assigned themselves to another gender. Their average age was 13.78 years (SD = 0.87). In the control group there were n = 117 adolescents who did not play like2be, of whom 57.3% were female, 38.5% were male, and 4.3% assigned themselves to another gender. Their average age was 13.62 years (SD = 0.61). The participants were surveyed at two measurement points (T1, T2). Between T1 (November 2021) and T2 (December 2021), there was an intervention phase of four weeks. During the intervention phase, the adolescents in the experimental groups played like2be twice for two entire lessons of 45 min each. During the two lessons, the teachers were present but did not offer any assistance; except for technical problems. The adolescents in the control group did not play like2be.
We found that the intervention with the serious game had a positive effect on expanding knowledge about occupations [19]. In this respect, students in the intervention groups could effectively expand their job-related knowledge with the serious game like2be (e.g., knowledge about the job, requirements, or job benefits). Their post-test scores (i.e., knowledge about occupations) were significantly higher than those of the control group.
Since recent research has shown that serious gaming can effectively promote learning on different levels, such as the acquisition of subject-specific learning content, but also the stimulation of factors that promote learning, we focus on the latter. In the present study we aim to describe more precisely to what extent certain factors (cognitive processes, flow, enjoyment, and subjectively perceived benefit) influenced the learning outcome with the serious game like2be. Therefore, we tested the following four hypotheses among those students who played like2be during the intervention:
H1. 
The serious game stimulates the cognitive processes and thus supports the expansion of knowledge about occupations.
H2. 
The serious game is highly enjoyable and thus supports the expansion of knowledge about occupations.
H3. 
The serious game stimulates the flow experience and thus supports the expansion of knowledge about occupations.
H4. 
The serious game leads to a high level of self-perceived benefit and thus supports the expansion of knowledge about occupations.

2.1. Operationalization of the Constructs

The study included two measurement points (T1, T2) in which the adolescents completed an online questionnaire. At both measurement points, we asked the participants how they would rate their knowledge of each occupation from like2be on a 6-point Likert Scale ranging from “nothing” to “a lot”. Since we found a significant group difference showing that the experimental groups learned more than the control group [19], we used the score from the post-test (T2) as the outcome variable for the data analysis.
Further, to evaluate the serious game like2be for its effectiveness, we created a new questionnaire based on the “Evaluation of the learning game AWWWARE” questionnaire [29] and the eGameFlow questionnaire [63]. We used items to measure eight of ten latent factors from the scale “Evaluation of the learning game AWWWARE”, developed by Iten and Petko [29] based on the EGameFlow Scale (Fu and colleagues [63]). Furthermore, we added items to the scale to measure the latent factor “Challenge” because of its high importance regarding the effectiveness of a serious game [63]. We used the three valid and reliable items with the highest factor loadings from the EGameFlow Scale. Our new scale “Effectiveness Scale for like2be” (see Table 1) consisted of nine subscales (factors): (1) Goal clarity (three items): the game objectives should be clear from the beginning and throughout the game. (2) Controlling the game (three items): game controls should be simple so that players can quickly navigate through the game and focus on its content or tasks. (3) Strategic approach (three items): the game should stimulate cognitive processes during play so that successful game strategies are developed. (4) Use of prior knowledge (three items): The game should be a game for everyone. In this respect, all players should be able to play successfully, regardless of their prior knowledge of the subject matter. (5) Flow during gameplay (three items): the game should lead the player into a state of immersion. (6) Enjoyment of the game (three items): the game should be entertaining and fun. (7) Challenge of the game (three items): the game should offer challenges that fit the player’s level of skills and the difficulty of these challenges should change in accordance with the increase in the player’s skill level. (8) Learning Outcome (four items): the game should increase the level of knowledge or skills of the players while meeting the game objectives. (9) Motivational Outcome (three items): the game should create motivation to explore a certain topic.

2.2. Scale Design

To investigate the factorial structure of the Effectiveness scale for like2be, we computed a confirmatory factor analysis (CFA) using the lavaan package in R. To determine whether the data were multivariate normally distributed, we applied the Mardia test for skewness and kurtosis [72]. Since both p-values were significant, we concluded that the data did not have a multivariate normal distribution. To correct for the violation of the multivariate normal distribution, we performed robust estimation of the model using the Satorra–Bentler adjustments [73,74] for all calculations.
First, we analyzed the basic model (Effectiveness scale for like2be) for its model fit. For this purpose, we evaluated the global fit, the local fit, and the parameter estimation. According to Hu and Bentler [74] the fit indices (see Table 2) suggest that the model did not fit the data.
Second, we analyzed the basic model (Effectiveness scale for like2be) for its model fit a second time, but this time we tested a one-factor model. Therefore, we added all 22 items to one factor. However, according to Hu and Bentler [74] the fit indices (see Table 3) suggest that the second model did not fit the data either.
Third, we reduced the basic model 1 (Effectiveness scale for like2be) to an adjusted model with the four subscales (factors): (1) Cognitive learning process, (2) Enjoyment of the game, (3) Flow during the game, and (4) Learning Outcome. The first subscale refers to competitive, challenging, problem-oriented gameplay and other mental processes that stimulate players’ cognition during their gameplay (e.g., goal clarity, gaming strategy, use of prior knowledge, learning by doing). In the context of serious gaming research, the stimulation of cognitive processes is considered to be particularly effective for learning because it can improve the attention performance, short-term memory and visual memory [51,53,54,55]. Since enjoyment of the game was attributed great importance [29,54,61,63], we created a second subscale including all items related to fun, entertainment, enjoyment or items that appeal to players on an emotional level. Although the flow experience was described as a component of motivation [64,75], the state of immersion is considered to be a very important component for the effectiveness of serious games [63] and we therefore created a separate third subscale for flow. Finally, serious games should increase the level of knowledge and skills of players while also meeting the games’ objectives to encourage the players to keep playing [63,65]. Consequently, learning outcome as the fourth subscale included all items related to self-perceived learning effect, achievement of game objectives, or learning progress.
Since the adjusted model was improved, but the fit indices still pointed to an insufficient model fit (see Table 4), we identified and excluded certain items (see items with (e) in Table 1) from the model based on the modification indices. Criteria for the exclusion of items were: items that lead to a large improvement in the model and (a) are indistinguishable from other items or ask the same question in different words, (b) can be assigned to more than one factor, or (c) correlate strongly with one or more error terms of other items. We also omitted all items of the initial factor “Controlling the game” because like2be is a point-and-click game. Hence, the game control did not present any obstacles, particularly not for adolescents.
Fourth, we created the final adjusted model, including the four subscales (factors): (1) Cognitive learning process (seven items, Cronbach’s α = 0.81), (2) Enjoyment of the game (four items, Cronbach’s α = 0.89), (3) Flow during the game (four items, Cronbach’s α = 0.80), and (4) Learning outcome (seven items, Cronbach’s α = 0.89). The fit indices of the final adjusted model (see Table 5) also showed that the model fit sufficiently. Thus, the CFI was >0.95, the RMSEA was <0.06 and statistically not significant, and the SRMR was <0.08, which, according to Hu and Bentler [74], is a good model fit. Also, in terms of parameter estimation, the model contained no negative variances, and the standardized loadings between the items and the factors were high throughout the model; the loadings within the factors were similarly high (see Table 6).

2.3. Data Analysis Methods

Again, using the lavaan package in R, we conducted a multiple linear regression and structural equation modeling (SEM) to analyze how the four factors identified above (cognitive learning process, enjoyment of the game, flow during the game, and learning outcome) predicted knowledge gain. In this process, the latent (exogenous or non-observable) factors are theoretical constructs that cannot be observed directly. Therefore, latent factors include observable/manifest (endogenous or dependent) variables. In our model, the four dimensions are latent factors (Cognitive learning process, Enjoyment of the game, Flow during the game, Learning outcome) emerged based on the 22 observed variables (C1-L7) (see Table 4).
With SEM, we performed a path analysis to (1) understand more precisely the correlation patterns among factors in our model, and (2) to show how much of the variation in the learning outcome of the intervention with like2be can be explained by the latent factors.

3. Results

Based on our CFA, we regarded clp (cognitive learning process), ejm (enjoyment of the game), flw (flow during the game), and loc (self-perceived learning outcome) as latent factors. To analyze to what extent the four latent factors influenced the increase in knowledge about occupations through playing the serious game like2be, we defined knowledge of occupations measured in the post-test as the outcome variable. To analyze to what extent the four latent factors influenced the increase in knowledge about occupations due to playing like2be, we defined the knowledge about all occupations from the serious game measured in the posttest as the outcome variable kaj (knowledge about jobs).
To determine the direction and strength of associations among the latent factors, we performed Pearson’s correlation. The correlation matrix (see Table 7) shows positive high and statistically significant correlations within the latent factors.
First, a multiple linear regression showed that the overall final adjusted model explained 17% of the variance in the outcome variable (F(4410) = 20.87, p < 0.001). Second, we analyzed the impact of the latent factors using SEM. The initial model fitting indices by using the Satorra–Bentler adjustments for the SEM were as follows: χ2 (df) = 1.284 (221), CFI = 0.960, RMSEA (p-value) = 0.043 (p = 0.978), SRMR = 0.044. Overall, the SEM model fitted the data well. In this context, the factor clp explained 14% of the outcome variable kaj (β = 0.43, EST = 13.97, SE = 6.11, z = 2.29). Additionally, the influence of the latent factor on the outcome variable was statistically significant (p = 0.022). However, the three remaining latent factors ejm (β = 0.12, EST = 1.78, SE = 1.94, z = 0.91, p = 0.361), flw (β = −0.23, EST = −3.91, SE = 3.62, z = −1.08, p = 0.280), and loc (β = 0.10, EST = 1.93, SE = 2.91, z = 0.66, p = 0.507) had lower factor loadings than clp, were not statistically significant, and therefore showed no significant effect on the outcome variable (see Figure 2).

4. Discussion

In this study, we emphasized the effectiveness of serious gaming for purposes of vocational orientation. In particular, we examined the extent to which cognitive, affective, and motivational factors have an impact on measured learning outcomes in the context of an intervention with the serious game like2be.
First, we developed a scale to evaluate the effectiveness of like2be. The basic model (Effectiveness Scale for like2be) included nine factors (see Table 1). Despite adopting all items from reliable and valid scales (Iten and Pekto [29]; Fu and colleagues [63]), our basic model was found to be inadequate in the course of the CFA. The fit indices did not indicate a good model fit. Consequently, we optimized the model. For this, we reduced our model to four dimensions based on current research. The first dimension included items in relation to measuring the extent of stimulation of cognitive processes (1st factor), which were considered to be particularly effective for learning [51,53,54,55]. The second dimension referred to items that measured the extent of enjoyment of the game (2nd factor), which were considered to be of great importance in terms of learning success [29,54,61,63]. The third dimension contained items measuring the flow experience (3rd factor), as state of immersion is considered to be a very important component in the effectiveness of serious games [63]. Finally, the fourth dimension included items measuring self-perceived learning outcome (4th factor), which motivates gamers to continue playing [63,65]. After reducing the basic model to four dimensions, we excluded a few items because they were either indistinguishable from other items or severely disrupted the model. Despite the fact that we could show with a CFA that our final adjusted model fits the data well, we should note that it deviates considerably from the factor models in past studies. One possible explanation for this could be that the serious games in the two studies we referred to were not comparable to the serious game like2be in terms of gameplay, subject matter, content, controls, etc. However, our final adjusted model is reliable and valid and is suitable for further evaluations of the effectiveness of serious games, especially in the field of vocational orientation.
Second, by applying regression analysis, we investigated the overall impact of the four latent factors on increase in knowledge about occupations. Our results indicate a statistically significant impact explaining 17% of variance. Despite the proportion of explained variance being rather small, cognitive, affective, and motivational factors turned out to positively influence knowledge acquisition and contribute to effective learning with serious games.
Third, we conducted a SEM to analyze the extent to which the four latent factors influence the measured increase in knowledge about occupations. Regarding H1, the results showed that stimulation of cognitive processes while playing like2be had a positive and statistically significant effect on increased knowledge about occupations. Therefore, we accept H1, that serious gaming stimulates cognitive processes and thus supports the expansion of knowledge about occupations. Regarding the importance of problem-based learning in gameplay [51], and game challenge, as well as the prospect of winning [52], our data analysis showed that the challenge of achieving the game objectives, the game strategies necessary to do so, and also prior knowledge, all had an impact on the expansion of knowledge. According to Sailer and Homner [53], collaboration and competition are important for cognitive stimulation. Although, like2be offers few possibilities for collaborative play, it includes competitive aspects because of the game time limit. Despite not evaluating effectiveness on attention capacity [54], short-term and visual memory performance [55], we conclude that the serious game like2be stimulates players’ cognition with its competitive, challenging, and problem-oriented gameplay during the game and therefore has a positive impact on knowledge enhancement.
Furthermore, the data analysis indicated that how much players enjoy playing like2be has no significant effect on increasing knowledge about occupations. Thus, we reject H2, that the serious game like2be is highly enjoyable and supports the expansion of knowledge about occupations. In accordance with Iten and Petko [29], we did not find a clear connection between enjoyment and learning outcome (i.e., increased knowledge) either. In line with Wouters and colleagues [50], we assume that like2be was not as entertaining as other commercial computer games and was therefore less motivating. Additionally, the lack of an exciting narrative or story in like2be with high exploration, challenge, or success aspects may have led to a lower impact of this serious game on learning outcomes [62].
Moreover, the results of SEM indicated that the flow experience while playing like2be had no significant effect on improving knowledge of occupations. Accordingly, we reject H3, that the serious game like2be stimulates the flow experience. Although, the serious game like2be provided intellectual competition in an entertaining way [22,23,24] and aspects such as comprehensible game goals, rules and gameplay, plus feasible challenges were given, it neither led to a GameFlow state [63,65] nor to a situation of complete absorption in the gaming activity mentioned by Csíkszentmihályi [64]. A possible explanation is offered by the study results relating to enjoyment. Thus, we assume that players did not experience a particularly high level of enjoyment while playing like2be for several reasons.
In terms of H4, the data analysis showed that the extent of subjectively experienced benefits through like2be had no significant effect on increasing knowledge of occupations. Therefore, we reject H4, that the serious game like2be leads to high level of self-perceived benefit. In this context, like2be did not contain in-game scaffolding [68] or did it provide players with a way to ask for feedback [51]. In this respect, like2be does not include in-game opportunities for additional reinforcement of the content, which would consolidate learning and help players realize that they are learning successfully through serious gaming. However, there is additional teaching material to deepen learning, following Wouters and colleagues [50], who mention that serious games are most effective for learning when supplemented with additional teaching materials. Furthermore, a set of analog activities for the serious game like2be already exists and includes reflection, modeling, collaboration, or personalization activities designed to support learners and help them continue to learn successfully [50,67,69]. The materials have even been favorably evaluated but were unfortunately not part of the current investigation.

5. Conclusions

Recent studies have shown that serious games can be used effectively for various educational purposes, in particular to promote the acquisition of specific technical knowledge, but also to enhance factors that can promote effective learning. In this study, we focused on the impact of the serious game like2be on factors conducive to learning, such as increasing cognitive performance and enjoyment of the game, stimulation of a flow experience and self-perceived benefits through serious gaming.
As a result of our data analysis, we conclude that like2be stimulates cognitive processes in players, which consequently has a positive effect on their expansion of knowledge about occupations. However, we were not able to demonstrate that the impact on enjoyment, flow experience, and self-perceived benefit through like2be promoted the expansion of knowledge about occupations. Indeed, these findings are in accordance with or can be explained by previous research. For example, enjoyment of a serious game was found to be very important for learning outcome [63], but to provide enjoyment, serious games must include an exciting narrative or story [62]. Additionally serious games must be as highly enjoyable as commercial computer games in order to enhance motivation and thus learning outcomes [50]. Since the fun factor of serious games is not considered competitive with commercial computer games for several reasons [50], it is not surprising that in their intervention study Iten and Pekto [29] found no impact of enjoyment on measured learning outcomes. Considering that enjoyment of the game is a relevant indicator of immersion in an activity [63,64,65], we assume that the perceived level of enjoyment while playing like2be was too low and therefore failed to promote a Flow experience. Further, studies have shown that when players had an in-game opportunity to request scaffolding [68] or feedback [51], learning processes with serious games were effectively enhanced. Due to the fact that like2be does not provide in-game scaffolding or feedback, it was not surprising that the data showed no impact on expansion of knowledge about occupations. Furthermore, Wouters and colleagues [50] have shown that additional instructional materials such as reflection, modeling, collaboration, or personalization [50,67,69] can enhance the impact of serious gaming. Although, such additional teaching materials have been developed and are available for like2be, their impact was not considered in the context of this study.
On the positive side, our scale (final adjusted model, see Table 6) can be used for future evaluations of the effectiveness of serious games, as it was found to be reliable and valid after testing with confirmatory factor analysis (CFA).
In conclusion, the serious game like2be stimulates cognitive processes through its competitive, challenging, and problem-oriented gameplay, consequently promoting effective learning. Thus, it represents an effective tool and should be considered for use in career choice classes.
For the future, we recommend intensifying research in the field of the effectiveness of serious games, especially in the field of vocational orientation. Regarding like2be, the effect of the additional teaching material on cognitive, affective and motivational factors conducive to learning should be analyzed. Furthermore, our data analysis shows that the game can be optimized in terms of enjoyment, flow experience, or self-perceived benefits. In this respect, like2be should be evaluated with a view to identifying opportunities to add more fun, flow experience indicators, or a scaffolding component to the gameplay. With regard to an optimization of like2be as well as the development of other serious games, we therefore recommend a stronger collaboration between game developers, educational experts and the target groups of players. We consider educational experts (e.g., teachers or educational scientists) as ideal contacts for developing specific scaffolding as well as an adequate and exciting narrative. Nevertheless, we suggest asking the target groups pf players (e.g., students) about which aspects make computer games exciting and appealing for them. The findings can be incorporated into the development of new or the optimization of existing serious games and thus generate a high degree of enjoyment and flow experience.

Author Contributions

Conceptualization, C.K., E.M. and A.K.D.; methodology, C.K., E.M. and A.K.D.; validation, C.K.; formal analysis, C.K. and A.K.D.; investigation, C.K., E.M. and A.K.D.; resources, C.K. and E.M.; data curation, C.K.; writing—original draft preparation, C.K.; writing—review and editing, C.K., E.M. and A.K.D.; visualization, C.K.; project administration, C.K. and E.M. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This study was reviewed and approved by the Ethics Committee and the Data Protection Officer of the University of Basel (date of approval 24 August 2021).

Informed Consent Statement

Informed consent was obtained from all subjects involved in this study.

Data Availability Statement

The data are not publicly available due to international data sharing agreements.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Yang, K.-H.; Chu, H.-C.; Chiang, L.-Y. Effects of a progressive prompting-based educational game on second graders’ mathematics learning performance and behavioral patterns. J. Educ. Technol. Soc. 2018, 21, 322–334. [Google Scholar]
  2. Makarova, E.; Lüthi, J.; Hofmann, M. Innovative Wege einer gendersensiblen Berufsorientierung. In Instrumente der Berufsorientierung; Pädagogische Praxis im wissenschaftlichen Diskurs; Brüggemann, T., Driesel-Lange, K., Weyer, C., Eds.; Waxmann: Münster, Germany, 2017; pp. 239–251. [Google Scholar]
  3. Breitlauch, L. Conceptual design for serious games regarding didactical and playfully requirements. In Applied Playfulness, Proceedings of the Vienna Games Conference 2011: Future and Reality of Gaming, Vienna, Austria, 21–23 October 2017; Wimmer, J., Mitgutsch, K., Rosenstingl, H., Eds.; New Academic Press: Vienna, Austria, 2012; pp. 91–97. [Google Scholar]
  4. Keller, C.; Doering, A.K.; Makarova, E. The Potential of Serious Games to Foster Learning among Children and Adolescents with Disabilities: A Systematic Review. Digit. Cult. Educ. 2021, 13, 6–36. [Google Scholar]
  5. Wastiau, P.; Kearney, C.; Van den Berghe, W. Games in School—How Are Digital Games Used in Schools? Full report.; European Schoolnet: Brussels, Belgium, 2009. [Google Scholar]
  6. Keller, C.; Makarova, E.; Döring, A.K. Förderung der Exploration im Berufswahlprozess mit Serious Games am Beispiel von like2be. In Entwicklungen und Perspektiven in der Berufsorientierung—Stand und Herausforderungen; Weyland, U., Ziegler, B., Driesel-Lange, K., Kruse, A., Eds.; Barbara Budrich: Opladen, Germany, 2021; pp. 135–155. [Google Scholar]
  7. Spangenberger, P.; Kapp, F.; Kruse, L.; Hartmann, M.; Narciss, S. Can a Serious Game Attract Girls to Technology Professions? Int. J. Gend. Sci. Technol. 2018, 10, 253–264. [Google Scholar]
  8. Spangenberger, P.; Draeger, I.; Kapp, F.; Kruse, L.; Matthes, N. Gendersensible Berufsorientierung mit einem Serious Game: Didaktische Einbettung des Spiels „Serena Supergreen“. In Digitale Lernwelten—Serious Games und Gamification: Didaktik, Anwendungen und Erfahrungen in der Beruflichen Bildung; Becker, W., Metz, M., Eds.; Springer Fachmedien Wiesbaden: Wiesbaden, Germany, 2022; pp. 127–146. [Google Scholar]
  9. Brown, S.D.; Lent, R.W. Career Development and Counseling: Putting Theory and Research to Work, 2nd ed.; John Wiley & Sons: Hoboken, NJ, USA, 2013. [Google Scholar]
  10. Herzog, W.; Makarova, E. Berufsorientierung als Copingprozess. In Berufsorientierung: Ein Lehr- und Arbeitsbuch; Brüggemann, T., Rahn, S., Eds.; Waxmann: Münster, Germany, 2020; pp. 73–82. [Google Scholar]
  11. Holland, J.L. Making Vocational Choices: A Theory of Vocational Personalities and Work Environments, 3rd ed.; Psychological Assessment Resources: Odessa, Ukraine, 1997. [Google Scholar]
  12. Swanson, J.L.; Schneider, M. Minnesota Theory of Work Adjustment. In Career Development and Counseling: Putting Theory and Research to Work; Brown, S.D., Lent, R.W., Eds.; John Wiley & Sons: Hoboken, NJ, USA, 2013; pp. 29–53. [Google Scholar]
  13. Super, D.E. A life-span, life-space perspective on convergence. In Convergence in Theories of Career Choice and Development: Implications for Science and Practice; Savickas, M.L., Lent, R.W., Eds.; Consulting Psychologists Press: Santa Clara, CA, USA, 1994; pp. 63–74. [Google Scholar]
  14. Savickas, M.L. Career Construction Theory and Practice. In Career Development and Counseling: Putting Theory and Research to Work; Brown, S.D., Lent, R.W., Eds.; John Wiley & Sons: Hoboken, NJ, USA, 2013; pp. 147–183. [Google Scholar]
  15. Gottfredson, L.S. Applying Gottfredson’s Theory of Circumscription and Compromise in Career Guidance and Counseling. In Career Development and Counseling: Putting Theory and Research to Work; Brown, S.D., Lent, R.W., Eds.; John Wiley & Sons: Hoboken, NJ, USA, 2005; pp. 71–127. [Google Scholar]
  16. Hartkopf, E. Berufswahlreife und Berufswahlkompetenz—Zwei Schlüsselbegriffe der Berufswahlforschung und der Berufsorientierungspraxis aus psychologischer und pädagogischer Perspektive. In Berufsorientierung. Ein Lehr-und Arbeitsbuch; Brüggemann, T., Rahn, S., Eds.; Waxmann: Münster, Germany, 2013; pp. 42–57. [Google Scholar]
  17. Kracke, B.; Schmitt-Rodermund, E. Adolescents’ Career Exploration in the Context of Educational and Occupational. In Navigating through Adolescence: European Perspectives; Nurmi, J.-E., Ed.; Routledge: New York, NY, USA, 2001; pp. 141–165. [Google Scholar]
  18. Neuenschwander, M.P.; Gerber, M.; Frank, N.; Rottermann, B. Schule und Beruf. Wege in die Erwerbstätigkeit; VS Verlag: Wiesbaden, Germany.
  19. Keller, C. Using the serious game like2be to effectively support adolescents in vocational orientation: A quasi-experimental intervention study (manuscript submitted for publication).
  20. Caillois, R. Man, Play, and Games; Free Press of Glencoe: New York, NY, USA, 1961. [Google Scholar]
  21. Dempsey, J.V.; Haynes, L.L.; Lucassen, B.A.; Casey, M.S. Forty Simple Computer Games and What They Could Mean to Educators. Simul. Gaming 2002, 33, 157–168. [Google Scholar] [CrossRef]
  22. Zyda, M. From visual simulation to virtual reality to games. Computer 2005, 38, 25–32. [Google Scholar] [CrossRef]
  23. Klopfer, E.; Osterweil, S.; Salen, K. Moving Learning Games Forward; The Education Arcade: Cambridge, MA, USA, 2009. [Google Scholar]
  24. Gotterbarn, D. Serious games: Learning why professionalism matters can be fun. ACM Inroads 2013, 4, 26–28. [Google Scholar] [CrossRef]
  25. Bogost, I. Persuasive Games; MIT Press: Cambridge, MA, USA, 2007; Volume 5. [Google Scholar]
  26. Boyle, E.; Connolly, T.M.; Hainey, T. The role of psychology in understanding the impact of computer games. Entertain. Comput. 2011, 2, 69–74. [Google Scholar] [CrossRef]
  27. Felicia, P. The Use of Computer Games in Education. In Handbook of Research on Improving Learning and Motivation through Educational Games; IGI Global: Hershey, PA, USA, 2011; pp. 29–50. [Google Scholar]
  28. Corti, K. Games-based Learning; a serious business application. Inf. De PixelLearning 2006, 34, 1–20. [Google Scholar]
  29. Iten, N.; Petko, D. Learning with serious games: Is fun playing the game a predictor of learning success? Br. J. Educ. Technol. 2016, 47, 151–163. [Google Scholar] [CrossRef] [Green Version]
  30. Pourabdollahian, B.; Taisch, M.; Kerga, E. Serious games in manufacturing education: Evaluation of learners’ engagement. Procedia Comput. Sci. 2012, 15, 256–265. [Google Scholar] [CrossRef] [Green Version]
  31. Cain, J.; Piascik, P. Are serious games a good strategy for pharmacy education? Am. J. Pharm. Educ. 2015, 79, 47. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Le Compte, A.; Elizondo, D.; Watson, T. A renewed approach to serious games for cyber security. In Proceedings of the 7th International Conference on Cyber Conflict: Architectures in Cyberspace, Tallinn, Estonia, 26–29 May 2015. [Google Scholar]
  33. Wilson, D.; Calongne, C.; Henderson, B. Gamification challenges and a case study in online learning. J. Online Learn. Res. Pract. 2015, 4, 26910. [Google Scholar] [CrossRef]
  34. Dicheva, D.; Dichev, C.; Agre, G.; Angelova, G. Gamification in education: A systematic mapping study. J. Educ. Technol. Soc. 2015, 18, 75–88. [Google Scholar]
  35. Willmott, T.; Russell-Bennett, R.; Drennan, J.; Rundle-Thiele, S. The Impact of Serious Educational Gameplay on Adolescent Binge Drinking Intentions: A Theoretically Grounded Empirical Examination. Health Educ. Behav. 2019, 46, 114–125. [Google Scholar] [CrossRef] [Green Version]
  36. Winskell, K.; Sabben, G.; Ondeng’e, K.; Odero, I.; Akelo, V.; Mudhune, V. A smartphone game to prevent HIV among young Kenyans: Household dynamics of gameplay in a feasibility study. Health Educ. J. 2019, 78, 595–606. [Google Scholar] [CrossRef]
  37. Byun, J.; Joung, E. Digital game-based learning for K-12 mathematics education: A meta-analysis. Sch. Sci. Math. 2018, 118, 113–126. [Google Scholar] [CrossRef]
  38. Tokac, U.; Novak, E.; Thompson, C.G. Effects of game-based learning on students’ mathematics achievement: A meta-analysis. J. Comput. Assist. Learn. 2019, 35, 407–420. [Google Scholar] [CrossRef]
  39. Talan, T.; Dogan, Y.; Batdi, V. Efficiency of digital and non-digital educational games: A comparative meta-analysis and a meta-thematic analysis. J. Res. Technol. Educ. 2020, 52, 474–514. [Google Scholar] [CrossRef]
  40. Hussein, M.H.; Ow, S.H.; Elaish, M.M.; Jensen, E.O. Digital game-based learning in K-12 mathematics education: A systematic literature review. Educ. Inf. Technol. 2022, 27, 2859–2891. [Google Scholar] [CrossRef]
  41. Chiu, Y.H.; Kao, C.W.; Reynolds, B.L. The relative effectiveness of digital game-based learning types in English as a foreign language setting: A meta-analysis. Br. J. Educ. Technol. 2012, 43, E104–E107. [Google Scholar] [CrossRef]
  42. Chen, M.H.; Tseng, W.T.; Hsiao, T.Y. The effectiveness of digital game-based vocabulary learning: A framework-based view of meta-analysis. Br. J. Educ. Technol. 2018, 49, 69–77. [Google Scholar] [CrossRef]
  43. Tsai, Y.L.; Tsai, C.C. A meta-analysis of research on digital game-based science learning. J. Comput. Assist. Learn. 2020, 36, 280–294. [Google Scholar] [CrossRef]
  44. Riopel, M.; Nenciovici, L.; Potvin, P.; Chastenay, P.; Charland, P.; Sarrasin, J.B. Un Impact of serious games on science learning achievement compared with more conventional instruction: An overview and a meta-analysis. Stud. Sci. Educ. 2020, 55, 169–214. [Google Scholar] [CrossRef] [Green Version]
  45. Hu, Y.; Gallagher, T.; Wouters, P.; van der Schaaf, M.; Kester, L. Game-based learning has good chemistry with chemistry education: A three-level meta-analysis. J. Res. Sci. Teach. 2022, 59, 1499–1543. [Google Scholar] [CrossRef]
  46. Lei, H.; Chiu, M.M.; Wang, D.; Wang, C.; Xie, T. Effects of Game-Based Learning on Students’ Achievement in Science: A Meta-Analysis. J. Educ. Comput. Res. 2022, 60, 1373–1398. [Google Scholar] [CrossRef]
  47. Huizinga, J. Homo Ludens; Routledge: London, UK, 1949. [Google Scholar]
  48. Heimlich, U. Einführung in die Spielpädagogik, 3rd ed.; UTB: Bad Heilbrunn, Germany, 2015. [Google Scholar]
  49. Retter, H. Spielzeug: Handbuch zur Geschichte und Pädagogik der Spielmittel; Beltz: Weinheim, Germany, 1979. [Google Scholar]
  50. Wouters, P.; van Nimwegen, C.; van Oostendorp, H.; van der Spek, E.D. A Meta-Analysis of the Cognitive and Motivational Effects of Serious Games. J. Educ. Psychol. 2013, 105, 249–265. [Google Scholar] [CrossRef] [Green Version]
  51. Mao, W.; Cui, Y.; Chiu, M.M.; Lei, H. Effects of Game-Based Learning on Students’ Critical Thinking: A Meta-Analysis. J. Educ. Comput. Res. 2022, 59, 1682–1708. [Google Scholar] [CrossRef]
  52. Jong, B.-S.; Lai, C.-H.; Hsia, Y.-T.; Lin, T.-W.; Lu, C.-Y. Using Game-Based Cooperative Learning to Improve Learning Motivation: A Study of Online Game Use in an Operating Systems Course. IEEE Trans. Educ. 2013, 56, 183–190. [Google Scholar] [CrossRef]
  53. Sailer, M.; Homner, L. The Gamification of Learning: A Meta-analysis. Educ. Psychol. Rev. 2020, 32, 77–112. [Google Scholar] [CrossRef] [Green Version]
  54. Gelsomini, M.; Garzotto, F.; Montesano, D.; Occhiuto, D. Wildcard: A wearable virtual reality storytelling tool for children with intellectual developmental disability. In Proceedings of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA, 16–20 August 2016. [Google Scholar]
  55. El Kah, A.; Lakhouaja, A. Developing effective educative games for Arabic children primarily dyslexics. Educ. Inf. Technol. 2018, 23, 2911–2930. [Google Scholar] [CrossRef]
  56. Khamparia, A.; Pandey, B.; Mishra, B.P. Effects of microworld game-based approach on neuromuscular disabled students learning performance in elementary basic science courses. Educ. Inf. Technol. 2020, 25, 3881–3896. [Google Scholar] [CrossRef]
  57. Gunn, K.C.M.; Delafield-Butt, J.T. Teaching Children with Autism Spectrum Disorder with Restricted Interests: A Review of Evidence for Best Practice. Rev. Educ. Res. 2016, 86, 408–430. [Google Scholar] [CrossRef] [Green Version]
  58. Schneier, J.; Taylor, N. Handcrafted Gameworlds: Space-Time Biases in Mobile Minecraft Play. New Media Soc. 2018, 20, 3420–3436. [Google Scholar] [CrossRef]
  59. Nebel, S.; Schneider, S.; Beege, M.; Kolda, F.; Mackiewicz, V.; Rey, G.D. You cannot do this alone! Increasing task interdependence in cooperative educational videogames to encourage collaboration. Educ. Technol. Res. Dev. 2017, 65, 993–1014. [Google Scholar] [CrossRef]
  60. Elliott, D. Levelling the playing field: Engaging disadvantaged students through game-based pedagogy. Lit. Learn. Middle Years 2014, 22, 34–40. [Google Scholar]
  61. Kosmas, P.; Ioannou, A.; Retalis, S. Moving Bodies to Moving Minds: A Study of the Use of Motion-Based Games in Special Education. Techtrends 2018, 62, 594–601. [Google Scholar] [CrossRef]
  62. Breien, F.S.; Wasson, B. Narrative categorization in digital game-based learning: Engagement, motivation & learning. Br. J. Educ. Technol. 2021, 52, 91–111. [Google Scholar]
  63. Fu, F.-L.; Su, R.-C.; Yu, S.-C. EGameFlow: A scale to measure learners’ enjoyment of e-learning games. Comput. Educ. 2009, 52, 101–112. [Google Scholar] [CrossRef]
  64. Csikszentmihalyi, M. Flow. The Psychology of Optimal Experience; Harper Perennial: New York, NY, USA, 1990. [Google Scholar]
  65. Sweetser, P.; Wyeth, P. GameFlow. Comput. Entertain. 2005, 3, 3. [Google Scholar] [CrossRef]
  66. Wronowski, M.; Urick, A.; Wilson, A.S.P.; Thompson, W.; Thomas, D.; Wilson, S.; Elizondo, F.J.; Ralston, R. Effect of a Serious Educational Game on Academic and Affective Outcomes for Statistics Instruction. J. Educ. Comput. Res. 2019, 57, 2053–2084. [Google Scholar] [CrossRef]
  67. Wouters, P.; van Oostendorp, H. A meta-analytic review of the role of instructional support in game-based learning. Comput. Educ. 2013, 60, 412–425. [Google Scholar] [CrossRef]
  68. Chen, C.H.; Law, V. Scaffolding individual and collaborative game-based learning in learning performance and intrinsic motivation. Comput. Hum. Behav. 2016, 55, 1201–1212. [Google Scholar] [CrossRef]
  69. Reinmann, G. Didaktisches Handeln. Die Beziehung zwischen Lerntheorien und Didaktischem Design. In L3T. Lehrbuch für Lernen und Lehren mit Technologien; Ebner, M., Schön, S., Eds.; epubli: Berlin, Germany, 2013. [Google Scholar]
  70. Deutschschweizer Erziehungsdirektoren-Konferenz (D-EDK). Berufliche Orientierung. In Lehrplan 21; Deutschschweizer Erziehungsdirektoren-Konferenz (D-EDK): Luzern, Switzerland, 2016; pp. 502–511. [Google Scholar]
  71. Makarova, E.; Keller, C. Berufswahl von Schülerinnen und Schülern mit Serious Games fördern. Das Spiel like2be und seine Wirksamkeit. Berufsbildung 2019, 14–16. [Google Scholar]
  72. Kankainen, A.; Taskinen, S.; Oja, H. On Mardia’s Tests of Multinormality; Birkhäuser: Basel, Switzerland, 2004; pp. 153–164. [Google Scholar]
  73. STATA. Satorra–Bentler Adjustments. 2022. Available online: https://www.stata.com/stata14/sem-satorra-bentler/ (accessed on 26 November 2022).
  74. Hu, L.T.; Bentler, P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equ. Model. A Multidiscip. J. 1999, 6, 1–55. [Google Scholar] [CrossRef]
  75. Urhahne, D. Sieben Arten der Lernmotivation. Psychol. Rundsch. 2008, 59, 150–166. [Google Scholar]
Education 13 00016 g001 550
Figure 1. The like2be Gameplay. (1) Applicant, (2) job or training position, (3) applicant’s CV, (4) specified time (game round).
Figure 1. The like2be Gameplay. (1) Applicant, (2) job or training position, (3) applicant’s CV, (4) specified time (game round).
Education 13 00016 g001
Education 13 00016 g002 550
Figure 2. SEM path model (final adjusted model). clp: kaj: Knowledge about jobs, Cognitive learning process, ejm: Enjoyment of the game, flw: Flow during the game, loc: Learning outcome, *** p < 0.001.
Figure 2. SEM path model (final adjusted model). clp: kaj: Knowledge about jobs, Cognitive learning process, ejm: Enjoyment of the game, flw: Flow during the game, loc: Learning outcome, *** p < 0.001.
Education 13 00016 g002
Table
Table 1. Effectiveness scale for like2be.
Table 1. Effectiveness scale for like2be.
FactorItem Content
Goal clarity (three items)“I understood the goal at the beginning of the like2be game.”
“I always had the goal in my head while playing the like2be game.”
“The goal was clear throughout the game.” (e)
Controlling the game (three items)“The game control of the like2be game was difficult.” (Reverse item) (e)
“I had to be very skilled to control the like2be game.” (Reverse item) (e)
“I learned to control the like2be game very quickly.” (e)
Strategic approach (three items)“While playing the like2be game, I thought carefully about whether or not I was placing job applicants in suitable positions.”
“While playing the like2be game, I didn’t bother with placing job applicants in suitable positions, I just tried everything.” (Reverse item) (e)
At the end of the like2be game, I reflected on why the job placements were suitable or not.
Use of prior knowledge (three items)“To play the like2be game, it was important to know a lot about jobs.”
“To place job applicants in suitable positions, I had to know a lot about jobs.” (e)
“To improve myself, I need to learn more about jobs.”
Flow during gameplay (three items)“While playing, I only thought about the like2be game.”
“While I was playing the like2be game, I forgot everything else around me.”
“While playing the like2be game, I did not notice how time passed.”
Enjoyment of the game (three items)“The like2be game was a lot of fun.”
“I want to play the like2be game again.”
“The like2be game was entertaining.”
Challenge of the game (three items)“My skills in the like2be game improved as I mastered the challenges.”
“The like2be game offered new challenges with a reasonable pace of play.”
“I enjoyed the like2be game without being bored or anxious.”
Learning Outcome (four items)“With the like2be Game I learned about new jobs.”
“With the like2be Game I gained knowledge about jobs.”.
“Because of the like2be Game, I’ve been thinking about the career choices of women and men.”
“Because of the like2be game, I’ve been thinking about what jobs suit me.”
Motivational Outcome (three items)“The like2be game enhanced my interest in the topic of career choice.”
“Because of playing the like2be game, I realized that I wanted to learn more about jobs.”
“Because of the like2be game, I will think more about my career choice.”
(e): Items excluded for the final adjusted model.
Table
Table 2. CFA fit indices of basic model 1.
Table 2. CFA fit indices of basic model 1.
Fit StatisticRobust Basic Model
Chi2 (df)χ2 = 1.270 (314); p < 0.001
CFI0.907
RMSEA (90% CI)0.066 (0.060; 0.072); p = 0.002
SRMR0.087
Chi2: Chi-square, CFI: Comparative Fit Index, RMSEA: Root Mean Square Error of Approximation, CI: Confidence Interval, SRMR: Standardized Root Mean Square Residual.
Table
Table 3. CFA fit indices of basic model 2.
Table 3. CFA fit indices of basic model 2.
Fit StatisticRobust Basic Model
Chi2 (df)χ2 = 1.306 (350); p < 0.001
CFI0.801
RMSEA (90% CI)0.092 (0.087; 0.098); p < 0.001
SRMR0.076
Chi2: Chi-square, CFI: Comparative Fit Index, RMSEA: Root Mean Square Error of Approximation, CI: Confidence Interval, SRMR: Standardized Root Mean Square Residual.
Table
Table 4. CFA fit indices of adjusted model.
Table 4. CFA fit indices of adjusted model.
Fit StatisticRobust Basic Model
Chi2 (df)χ2 = 1.289 (344); p < 0.001
CFI0.873
RMSEA (90% CI)0.074 (0.068; 0.080); p < 0.001
SRMR0.069
Chi2: Chi-square, CFI: Comparative Fit Index, RMSEA: Root Mean Square Error of Approximation, CI: Confidence Interval, SRMR: Standardized Root Mean Square Residual.
Table
Table 5. CFA fit indices of final adjusted model.
Table 5. CFA fit indices of final adjusted model.
Fit StatisticFinal Adjusted Model
Chi2 (df)p = 1.311 (203); p < 0.001
CFI0.959
RMSEA (90% CI)0.044 (0.038; 0.050); p = 0.943
SRMR0.045
Chi2: Chi-square, CFI: Comparative Fit Index, RMSEA: Root Mean Square Error of Approximation, CI: Confidence Interval, SRMR: Standardized Root Mean Square Residual.
Table
Table 6. Final adjusted model.
Table 6. Final adjusted model.
FactorItem No.Item ContentMean, SDCoefficientStandard Errorz-Valuep-valueFactor Loading
Cognitive learning process
(seven items, α = 0.81)		C1“I understood the goal at the beginning of the like2be game.”M = 3.60, SD = 1.180.470.076.99<0.0010.40
C2“I always had the goal in my head while playing the like2be game.”M = 2.98, SD = 1.160.790.0516.39<0.0010.68
C3“While playing the like2be game, I thought carefully about whether or not I was placing job applicants in suitable positions.”M = 3.04, SD = 1.160.770.0514.62<0.0010.66
C4At the end of the like2be game, I reflected on why the placements of jobs were suitable or not.M = 2.66, SD = 1.180.850.0518.88<0.0010.72
C5“To play the like2be game, it was important to know a lot about jobs.”M = 2.99, SD = 1.190.600.0610.09<0.0010.50
C6“To improve myself, I need to learn more about jobs.”M = 2.92, SD = 1.180.690.0612.26<0.0010.59
C7“My skills in the like2be game improved as I mastered the challenges.”M = 2.87, SD = 1.140.830.0419.01<0.0010.73
Enjoyment of the game
(four items, α = 0.89)		E1“The like2be game was a lot of fun.”M = 3.06, SD = 1.241.060.0424.56<0.0010.86
E2“I want to play the like2be game again.”M = 2.78, SD = 1.341.080.0523.24<0.0010.81
E3“The like2be game was entertaining.”M = 3.01, SD = 1.251.070.0426.28<0.0010.85
E4“I enjoyed the like2be game without being bored or anxious.”M = 2.96, SD = 1.230.940.0519.80<0.0010.77
Flow during the game
(four items, α = 0.80)		Fl1“While playing, I only thought about the like2be game.”M = 2.70, SD = 1.260.890.0518.00<0.0010.71
Fl2“While I was playing the like2be game, I forgot everything else around me.”M = 2.39, SD = 1.170.840.0517.69<0.0010.72
Fl3“While playing the like2be game, I didn’t notice how time passed.”M = 2.79, SD = 1.260.920.0519.01<0.0010.73
Fl4“The like2be game offered new challenges with a reasonable pace of play.”M = 2.77, SD = 1.150.780.0515.41<0.0010.68
Learning outcome
(seven items, α = 0.89)		L1“With the like2be Game I learned about new jobs.”M = 3.17, SD = 1.240.780.0514.49<0.0010.63
L2“With the like2be Game I gained knowledge about jobs.”.M = 2.80, SD = 1.210.840.0517.58<0.0010.70
L3“Because of the like2be Game, I’ve been thinking about the career choices of women and men.”M = 2.34, SD = 1.190.810.0515.59<0.0010.68
L4“Because of the like2be game, I’ve been thinking about what jobs suit me.”M = 2.65, SD = 1.210.920.0519.67<0.0010.76
L5“The like2be game enhanced my interest in the topic of career choice.”M = 2.99, SD = 1.190.940.0520.66<0.0010.79
L6“Because playing the like2be game, I realized that I wanted to learn more about jobs.”M = 2.63, SD = 1.210.970.0423.73<0.0010.81
L7“Because of the like2be game, I will think more about my career choice.”M = 2.64, SD = 1.220.970.0421.98<0.0010.80
Table
Table 7. Correlation matrix of latent factors.
Table 7. Correlation matrix of latent factors.
clpejmflwloc
clp1.000
ejm0.802 ***1.000
flw0.866 ***0.821 ***1.000
loc0.880 ***0.737 ***0.821 ***1.000
clp: Cognitive learning process, ejm: Enjoyment of the game, flw: Flow during the game, loc: Learning outcome, *** p < 0.001.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Keller, C.; Döring, A.K.; Makarova, E. Factors Influencing the Effectiveness of Serious Gaming in the Field of Vocational Orientation. Educ. Sci. 2023, 13, 16. https://doi.org/10.3390/educsci13010016

AMA Style

Keller C, Döring AK, Makarova E. Factors Influencing the Effectiveness of Serious Gaming in the Field of Vocational Orientation. Education Sciences. 2023; 13(1):16. https://doi.org/10.3390/educsci13010016

Chicago/Turabian Style

Keller, Christopher, Anna K. Döring, and Elena Makarova. 2023. "Factors Influencing the Effectiveness of Serious Gaming in the Field of Vocational Orientation" Education Sciences 13, no. 1: 16. https://doi.org/10.3390/educsci13010016

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop