Technological development is advancing rapidly with considerable impact on education worldwide. One such advance in the educational field is learning through the use of serious games. Their implementation in the classroom not only enriches conventional teaching methods but has also become a popular research topic [1
]. Serious games contain major pedagogical potential, as they have been explicitly developed for learning purposes [2
]. They have been used for a number of different educational purposes [4
], including in the field of vocational orientation [6
Vocational orientation acquires special importance towards the end of compulsory schooling as adolescents need to decide which career they want to pursue. Career choice is a major biographical event for young people as it strongly relates to individual development and life experience [9
]. Furthermore, it is embedded in a personal and multi-layered career orientation process that begins in early childhood [10
]. In a career choice process, adolescents strive for a professional career that fits particularly well with their career-related personal characteristics. In the context of career research, the level of fit between these characteristics and those of the professional environment (occupational fit) indicates the probability of professional satisfaction, commitment and career stability [11
]. However, occupational fit is a complex construct. It emerges and is transformed through interaction with the environment [12
] and makes career orientation a lifelong process.
From early childhood, young individuals begin to develop their own personality, self-concept [13
], and role in their social environment [14
]. In the process, different interests, skills, competencies, and talents emerge, as well as values, strengths, and limitations. These are important indicators for the identification of career-related personality traits and for the formation of a career self-concept with which to find a fitting entry into the occupational world [13
]. However, children also learn about different professions at an early age. They recognize occupational activities and functions and learn to distinguish between them. Moreover, in interaction with their social–cultural environment, they observe not only who is typically employed in certain professions, but also what prestige is ascribed to certain professions. In consequence, the children develop their first occupational stereotypes (e.g., occupational gender roles), which influence their future career orientation [15
Because career choice is closely related to individual development and can be influenced by a variety of factors, it is important that young learners engage in career exploration. Research in the field of vocational education and training (VET) shows that their transition into the professional world is particularly successful when they are intensively engaged in in their own career exploration process [16
]. For this reason, the serious game like2be
was developed so that adolescents can learn about professions in a playful way and thus broaden their own career horizons [2
Since serious gaming offers an innovative alternative to conventional learning methods, empirical research in this research field has been intensified in recent years [1
]. To date, however, there is a lack of empirical evidence on effectiveness of serious games for career orientation purposes. To overcome this research gap, we conducted a quasi-experimental intervention study in order to discover whether like2be
effectively supports learning in the context of career choice. The data indicated that the integration of the serious game like2be
into school lessons can effectively support adolescents in acquiring knowledge about professions, thereby broadening their career horizons [19
In the present study, we applied a confirmatory factor analysis (CFA) and calculated a structural equation model (SEM) to provide a differentiated answer to the following research question: To what extent do cognitive, affective, and motivational factors influence learning with the serious game like2be?
1.1. Serious Games
The term serious game covers a wide range of attributes. First, a serious game is basically a game, and represents a series of voluntary and enjoyable activities in which players are involved. Furthermore, it is governed by constraints or rules, and it involves certain goals and possibilities for achieving these goals through moves or actions [20
]. Second, serious games are usually developed by experts explicitly for a specific target group and their abilities and needs, and since concrete learning objectives as well as media-didactic models are taken into account during development, they lead to an entertaining mental contest and are able to stimulate learning processes effectively [3
Although serious games do provide entertainment, their main purpose is learning [22
]. As such, they are primarily used for educational purposes [22
], but are occasionally used with the aim of supporting attitude and behavior change, e.g., to discourage smoking or promote recycling [25
]. The term Serious Game also includes the sub-category Digital Game-based Learning (DGBL), which refers to an innovative approach to learning skill acquisition and training through computer games (e.g., serious computer games) and therefore has high educational value and potential [26
]. However, it should be noted that the two terms (Serious Computer Gaming, DGBL) are sometimes used synonymously [28
]. In this paper, the terms Serious Game and Serious Gaming are used to refer exclusively to computer-based games.
1.2. Serious Gaming and Subject Learning
Serious Gaming has become a popular research topic. In particular, the past few years have seen research into whether serious games benefit learning in specific subject areas. Although studies have demonstrated that serious games can be used effectively for various “exotic” educational purposes, such as to reduce school phobia [5
], to raise students’ awareness of Internet dangers [29
], for vocational training purposes [30
], particularly in commercially oriented businesses [34
], or for addiction and disease prevention [35
], serious gaming has principally been analyzed for its effectiveness in the context of subject-related learning.
For example, Byun and Joung [37
] showed that serious games significantly benefited the learning of mathematics among students in grades K-12, although the effect was considered small and the data analysis implied that there may be other options for learning mathematics more effectively than with serious gaming. Likewise, Tokac and colleagues [38
] mentioned that serious gaming leads to small but significantly higher learning gains for students in PreK-12th-grade compared to traditional instructional methods. Talan and colleagues [39
] similarly described the effect of serious gaming on mathematics learning as moderate, but, unlike in previous studies, they included outcomes from preschool to university in the analysis. Further, in a currently published review, Hussein and colleagues [40
] also reported that serious gaming positively impacted mathematics learning in K-12 levels. Although the studies reviewed focus on a wide range of mathematics-related topics, most studies were limited to the effect of serious gaming on learning arithmetic operations.
The effectiveness of serious games for language learning has also been addressed in various studies. Chiu and colleagues [41
] found that serious gaming leads to a medium positive effect size with regard to foreign language learning. They noted that drill and practice games result in a small positive learning effect, while meaningful and engaging games result in a large positive effect size. Talan and colleagues [39
] also showed a large effect of serious gaming on language learning. They therefore suggest that second language learning is a promising area in which serious gaming could be more effective than traditional media for students from preschool to university. Similarly, Chen and colleagues [42
] mentioned large effects of serious gaming on language learning, especially on vocabulary acquisition. In this regard, a gaming approach appears to be superior to conventional learning methods. Similarly to a previous study, their results suggest that the effects of serious gaming on language learning depend on the characteristics of the game design. In particular, adventure games seem to be more challenging for players and thus prove more effective for learning outcomes.
As in learning math and language, research has shown that serious games have a positive impact on science learning achievements. For example, Tsai and Tsai [43
] compared science learning using conventional activities with learning supported by playing serious science games. They report that compared to conventional instruction in science classrooms, students learn significantly better with serious games in science class. Also, Riopel and colleagues [44
] showed that serious gaming is an effective alternative for science learning. Especially in terms of knowledge acquisition, knowledge learning and knowledge retention, was serious gaming shown to outperform learning with more conventional methods under conditions of similar time investment or engagement. Likewise, Hu and colleagues [45
] describe science learning through serious gaming as particularly effective. They also note that serious gaming had a positive effect on cognition, emotion, motivation, and retention and thus led to learning effects. Finally, Lei and colleagues [46
] also examined the effects of learning with serious games compared with traditional teaching methods on the academic achievement of students from elementary school through university. They found that students learned substantially more with the use of serious games than under traditional instruction. As possible moderation effects, the authors consider an increase in student autonomy or motivation.
Overall, the state of research shows that serious gaming is used for different purposes in the school context and supports learning, especially subject learning (e.g., mathematics, language or science). Furthermore, it shows that serious gaming can be used to create conditions conducive to learning.
1.3. Serious Gaming and Factors That Promote Learning
The cultural historian Johan Huizinga noted that the human species was once called Homo Sapiens
. When man himself realized that he was probably not as reasonable as assumed, Homo Faber
was created, man the maker. However, since humans have always also been playful beings, the human species, in addition to Homo Sapiens and Homo Faber, can also justifiably be labeled Homo Ludens
, man the player [47
]. Archaeological research has revealed that mankind engaged in play thousands of years ago. Mostly natural materials such as stones, fruit, grains, wood, and later bones were used for different forms of games. Excavations also confirm that the ball, the doll, and toy animals are among the oldest objects of play. Furthermore, ancient games such as nine men’s morris or chess are still played with great fascination today [48
]. Moreover, it is emphasized by historians, e.g., Retter [49
], that as early as approx. 5000 B.C. the peoples of present-day China, India, Persia, Egypt and Greece began to think about playful activities and their importance in education. Consequently, it can be assumed that humans have always had a marked tendency to play games, which gives games and play a considerable potential.
However, the potential of serious games goes far beyond the historical perspective. On the one hand, serious games fit in with trends of the current age, and on the other, they are exclusively available on devices (e.g., smart devices) that have great influence on people’s everyday lives. Concerning the professional development of serious games, many studies have reported that serious gaming has a positive effect on specific learning of subject knowledge among learners of different ages. Other studies have focused less on the learning outcome (i.e., subject related knowledge), but instead on factors that can promote effective learning [4
], such as an increase in cognitive performance or the stimulation of flow experience, of emotions (e.g., enjoyment), as well as of self-perceived motivating aspects (e.g., learning benefits through serious gaming).
1.3.1. Stimulation of Cognitive Processes with Serious Gaming
In a meta-analysis on serious gaming and its influence on cognitive processes and motivation, Wouters and colleagues [50
] found that serious games are more effective in learning and retention, but no more motivating than conventional instruction methods. However Mao and colleagues [51
] mention that serious gaming is particularly effective because it often focuses on problem-based learning, where players have to try different strategies to solve problems in order to progress within the game. Additionally, Jong and colleagues [52
] found that serious gaming has a positive learning impact when players perceive the game as a challenging task that also offers them the prospect of winning. Furthermore, Sailer and Homner [53
] find that gamified learning methods have significant but small overall effects on cognitive, motivational, and behavioral outcomes. Although they point out that gamified learning methods and serious gaming share the same focus on learning value beyond entertainment, but they differ in nature. They also add that collaborative and competitive aspects in learning games are of great importance for cognitive stimulation. Moreover, serious games have been effectively used to increase players’ cognitive performance level in terms of fostering attention capacity [54
], enhancing short-term and visual memory performance [55
], and reducing the fear of learning failure, thus having a positive effect on learning motivation [56
All studies showed that serious gaming stimulates cognitive processes and that this has a positive effect on learning. Here, the focus is on increasing motivation. A similar relationship exists between enjoyment and effective learning.
1.3.2. Enjoyment through Serious Gaming
Wouters and colleagues [50
] mention that it is generally assumed that serious games have a similar motivational appeal, such as high entertainment value, much as commercial computer games have. Nevertheless, they add that serious games cannot compete with commercial computer games in terms of gameplay, game content and game design and therefore have a less motivating effect. This is especially evident in the sandbox game MineCraft. Although it is not a serious game by definition, considerable effort has been put into the development of an Educational Edition in recent years. This edition was developed by educators for educators and includes innumerable possibilities to use MineCraft in a school context in a way that enhances learning processes. Owing to the Educational Edition, MineCraft can be understood as a kind of serious gaming. In various studies, MineCraft is said to have great educational potential, which is related to its high level of enjoyment [57
]. One indicator for this is its open, creative gameplay which encourages exploration and learning—even requires it [60
Moreover, self-confidence and self-efficacy are increased through serious gaming [56
], which is strongly related to the enjoyment of playing serious games, and which contributes to motivational or behavioral learning outcomes [54
]. Although Breien and Wasson [62
] consider the effect of serious games on learning outcomes to be positive in their review, a narrative or story included in the gameplay was found to be a key aspect of the effectiveness of serious gaming. In particular, they identified four narrative or story-related possibilities that increase the positive impact of serious games on enjoyment, engagement, and, consequently, learning: a virtual, quest-based game landscape that can be explored in-game, changeable objects used to overcome challenges and achieve goals, and game-relevant avatars whose own story makes an exciting contribution to the game story, or the integration of real, significant events in human history into the game story.
According to Iten and Petko [29
], however, although some scientific evidence suggests that serious gaming may enhance motivation and positive emotions (especially enjoyment), which are strong factors in positive learning outcomes, connections between fun and learning have not yet been fully analyzed by empirical research. They found no clear connection between fun and learning success in their intervention study evaluating the effectiveness of a serious game in Swiss schools. Yet their data showed that greater enjoyment of serious gaming led to greater interest in the learning content, and so they concluded that serious gaming is an effective alternative for introducing a new topic and increasing motivation during the learning process [29
Thus, studies have shown that serious games can be fun and entertaining and that they consequently promote learning. Nevertheless, it is assumed that they must provide players with a very high level of enjoyment in order to be effective for learning. Another factor related to effective learning is the experience of flow.
1.3.3. Stimulation of Flow Experience with Serious Gaming
Serious gaming can foster intellectual competition in an entertaining way [22
], and because it is an act of doing (i.e., learning by doing) it contributes to a flow experience. If certain factors are present during the game (such as comprehensible game goals, rules and gameplay, feasible challenges, player concentration, or feedback or assistance), players can fall into a flow state [63
]. According to Csíkszentmihályi [64
], this state is a situation of complete absorption or engagement in an activity and is of great importance to learning processes. If learners additionally experience a particularly high level of enjoyment while playing a serious game, they can fall into a GameFlow state [65
]. Fu and colleagues [63
] assume that players in a GameFlow state increase their motivation and engagement, which positively stimulates their learning process. Finally, Wronowski and colleagues [66
] reported that students who used a serious game (Deadly Distribution) for the purpose of learning statistics were highly absorbed during gameplay and showed higher levels of engagement and ultimately interest in the subject of statistics than those who used conventional methods.
For those who are completely immersed in an activity, totally focused and exclusively engaged, we talk about the mental sensation of flow. In the context of learning, this creates an environment conducive to learning. Research has shown that serious games can lead to such flow experiences. In addition to stimulating the flow experience, serious games can have an impact on self-perceived benefit through the activity.
1.3.4. Self-Perceived Benefit through Serious Gaming
Although serious games effectively promote learning, the use of supplemental materials appears to further increase learning outcome. Wouters and colleagues [50
] noted that serious games were most effective for learning processes when supplemented with other teaching materials rather than when used as the sole teaching method. In a learning process, serious games in combination with specific didactic support (e.g., reflection, modeling, collaboration, modality, feedback, or personalization) led to well-structured prior knowledge that helped learners to build on and continue to learn successfully [50
]. Similarly, Chen and Law [68
] mentioned the benefit of additional support. They analyzed the effectiveness of in-game scaffolding in the serious gaming process and concluded that with additional scaffolding, the positive effect of serious gaming on student motivation and learning performance was significantly enhanced. Moreover, for Mao and colleagues [51
], additional support also represents an important aspect of effective learning with serious games. If players receive beneficial feedback while gaming or afterwards, it helps them to better understand and reflect on information use as well as on the decisions made in the serious gaming process, which ultimately supports learning. In order to effectively support learning processes and make learning effects visible, learning options such as serious games must be methodologically and didactically embedded in the school context [69
In other words, serious games can support learning processes, but purposeful application and the use of additional learning materials can strengthen the learning effect. It appears to be important that additional deepening of the subject matter of the serious games consolidates learning on the one hand, and on the other, the players come to recognize that they can learn successfully through serious gaming.
1.4. like2be—A Serious Game for Vocational Orientation
In school lessons on “Vocational Orientation”, adolescents at the Swiss secondary level are specifically stimulated for an intensive and individual vocational exploration phase, the aim of which is to help them succeed in making the transition to the world of work [70
]. An innovative possibility for exploration in the context of career choice is the serious game like2be
was developed as part of a research project supported by the Swiss National Science Foundation (SNSF) with the involvement of various experts. It is a web-based online game and can be played for free in German, French and Italian (www.like2be.ch
). In addition, this serious game was developed specifically for adolescents in their career orientation process who will have to make an initial career choice decision by the end of compulsory schooling [2
, young people can expand their career choice horizons in a playful way. It is a simple point-and-click game without narrative. As players, they take on the role of a personnel agent who must place applicants in suitable jobs or training positions based on their application folders and CVs within a specified time (see Figure 1
). A suitable placement is followed by promotion at the end of the game round and the level of difficulty increases (i.e., more vacant jobs to choose from). In the case of unsuitable placements, the player is threatened with dismissal from the virtual job agency [2
includes different jobs or training positions, a variety of cartoon-style game characters (avatars), and various personal profiles (application folders and CVs). The jobs or training positions integrated into like2be are taken from the official Swiss information portal for career, study, and career guidance (www.berufsberatung.ch
). The serious game like2be
does not contain a specific roster. Instead, a randomization mechanism ensures that all application folders and CVs occurring in the game are randomly assigned to a game character (avatar) each time the game is started. Consequently, players can thereby only play successfully if they compare skills, abilities, and individual wishes of applicants with the job jobs or training positions offer [2
2. Study Design & Methodology
In 2021, we conducted a quasi-experimental intervention study with adolescents at the secondary school level to analyze the extent to which like2be can broaden young adults’ career choice horizons. The total sample included n = 809 adolescents from German-speaking Switzerland. Of these, 49.4% were female, 48.2% were male, 2% assigned themselves to another gender. Their average age was 13.77 years (SD = 0.82). After incomplete data were excluded, a final sample including 532 adolescents remained. Of these, n = 415 were in the intervention groups and played like2be in the classroom; 48.7% were female, 50.1% were male, and 1.2% assigned themselves to another gender. Their average age was 13.78 years (SD = 0.87). In the control group there were n = 117 adolescents who did not play like2be, of whom 57.3% were female, 38.5% were male, and 4.3% assigned themselves to another gender. Their average age was 13.62 years (SD = 0.61). The participants were surveyed at two measurement points (T1, T2). Between T1 (November 2021) and T2 (December 2021), there was an intervention phase of four weeks. During the intervention phase, the adolescents in the experimental groups played like2be twice for two entire lessons of 45 min each. During the two lessons, the teachers were present but did not offer any assistance; except for technical problems. The adolescents in the control group did not play like2be.
We found that the intervention with the serious game had a positive effect on expanding knowledge about occupations [19
]. In this respect, students in the intervention groups could effectively expand their job-related knowledge with the serious game like2be
(e.g., knowledge about the job, requirements, or job benefits). Their post-test scores (i.e., knowledge about occupations) were significantly higher than those of the control group.
Since recent research has shown that serious gaming can effectively promote learning on different levels, such as the acquisition of subject-specific learning content, but also the stimulation of factors that promote learning, we focus on the latter. In the present study we aim to describe more precisely to what extent certain factors (cognitive processes, flow, enjoyment, and subjectively perceived benefit) influenced the learning outcome with the serious game like2be. Therefore, we tested the following four hypotheses among those students who played like2be during the intervention:
The serious game stimulates the cognitive processes and thus supports the expansion of knowledge about occupations.
The serious game is highly enjoyable and thus supports the expansion of knowledge about occupations.
The serious game stimulates the flow experience and thus supports the expansion of knowledge about occupations.
The serious game leads to a high level of self-perceived benefit and thus supports the expansion of knowledge about occupations.
2.1. Operationalization of the Constructs
The study included two measurement points (T1
) in which the adolescents completed an online questionnaire. At both measurement points, we asked the participants how they would rate their knowledge of each occupation from like2be
on a 6-point Likert Scale ranging from “nothing” to “a lot”. Since we found a significant group difference showing that the experimental groups learned more than the control group [19
], we used the score from the post-test (T2
) as the outcome variable for the data analysis.
Further, to evaluate the serious game like2be
for its effectiveness, we created a new questionnaire based on the “Evaluation of the learning game AWWWARE” questionnaire [29
] and the eGameFlow questionnaire [63
]. We used items to measure eight of ten latent factors from the scale “Evaluation of the learning game AWWWARE”, developed by Iten and Petko [29
] based on the EGameFlow Scale (Fu and colleagues [63
]). Furthermore, we added items to the scale to measure the latent factor “Challenge” because of its high importance regarding the effectiveness of a serious game [63
]. We used the three valid and reliable items with the highest factor loadings from the EGameFlow Scale. Our new scale “Effectiveness Scale for like2be
” (see Table 1
) consisted of nine subscales (factors): (1) Goal clarity (three items): the game objectives should be clear from the beginning and throughout the game. (2) Controlling the game (three items): game controls should be simple so that players can quickly navigate through the game and focus on its content or tasks. (3) Strategic approach (three items): the game should stimulate cognitive processes during play so that successful game strategies are developed. (4) Use of prior knowledge (three items): The game should be a game for everyone. In this respect, all players should be able to play successfully, regardless of their prior knowledge of the subject matter. (5) Flow during gameplay (three items): the game should lead the player into a state of immersion. (6) Enjoyment of the game (three items): the game should be entertaining and fun. (7) Challenge of the game (three items): the game should offer challenges that fit the player’s level of skills and the difficulty of these challenges should change in accordance with the increase in the player’s skill level. (8) Learning Outcome (four items): the game should increase the level of knowledge or skills of the players while meeting the game objectives. (9) Motivational Outcome (three items): the game should create motivation to explore a certain topic.
2.2. Scale Design
To investigate the factorial structure of the Effectiveness scale for like2be
, we computed a confirmatory factor analysis (CFA) using the lavaan package in R. To determine whether the data were multivariate normally distributed, we applied the Mardia test for skewness and kurtosis [72
]. Since both p
-values were significant, we concluded that the data did not have a multivariate normal distribution. To correct for the violation of the multivariate normal distribution, we performed robust estimation of the model using the Satorra–Bentler adjustments [73
] for all calculations.
First, we analyzed the basic model (Effectiveness scale for like2be
) for its model fit. For this purpose, we evaluated the global fit, the local fit, and the parameter estimation. According to Hu and Bentler [74
] the fit indices (see Table 2
) suggest that the model did not fit the data.
Second, we analyzed the basic model (Effectiveness scale for like2be
) for its model fit a second time, but this time we tested a one-factor model. Therefore, we added all 22 items to one factor. However, according to Hu and Bentler [74
] the fit indices (see Table 3
) suggest that the second model did not fit the data either.
Third, we reduced the basic model 1 (Effectiveness scale for like2be
) to an adjusted model with the four subscales (factors): (1) Cognitive learning process, (2) Enjoyment of the game, (3) Flow during the game, and (4) Learning Outcome. The first subscale refers to competitive, challenging, problem-oriented gameplay and other mental processes that stimulate players’ cognition during their gameplay (e.g., goal clarity, gaming strategy, use of prior knowledge, learning by doing). In the context of serious gaming research, the stimulation of cognitive processes is considered to be particularly effective for learning because it can improve the attention performance, short-term memory and visual memory [51
]. Since enjoyment of the game was attributed great importance [29
], we created a second subscale including all items related to fun, entertainment, enjoyment or items that appeal to players on an emotional level. Although the flow experience was described as a component of motivation [64
], the state of immersion is considered to be a very important component for the effectiveness of serious games [63
] and we therefore created a separate third subscale for flow. Finally, serious games should increase the level of knowledge and skills of players while also meeting the games’ objectives to encourage the players to keep playing [63
]. Consequently, learning outcome as the fourth subscale included all items related to self-perceived learning effect, achievement of game objectives, or learning progress.
Since the adjusted model was improved, but the fit indices still pointed to an insufficient model fit (see Table 4
), we identified and excluded certain items (see items with (e)
in Table 1
) from the model based on the modification indices. Criteria for the exclusion of items were: items that lead to a large improvement in the model and (a) are indistinguishable from other items or ask the same question in different words, (b) can be assigned to more than one factor, or (c) correlate strongly with one or more error terms of other items. We also omitted all items of the initial factor “Controlling the game” because like2be
is a point-and-click game. Hence, the game control did not present any obstacles, particularly not for adolescents.
Fourth, we created the final adjusted model, including the four subscales (factors): (1) Cognitive learning process (seven items, Cronbach’s α = 0.81), (2) Enjoyment of the game (four items, Cronbach’s α = 0.89), (3) Flow during the game (four items, Cronbach’s α = 0.80), and (4) Learning outcome (seven items, Cronbach’s α = 0.89). The fit indices of the final adjusted model (see Table 5
) also showed that the model fit sufficiently. Thus, the CFI was >0.95, the RMSEA was <0.06 and statistically not significant, and the SRMR was <0.08, which, according to Hu and Bentler [74
], is a good model fit. Also, in terms of parameter estimation, the model contained no negative variances, and the standardized loadings between the items and the factors were high throughout the model; the loadings within the factors were similarly high (see Table 6
2.3. Data Analysis Methods
Again, using the lavaan package in R, we conducted a multiple linear regression and structural equation modeling (SEM) to analyze how the four factors identified above (cognitive learning process, enjoyment of the game, flow during the game, and learning outcome) predicted knowledge gain. In this process, the latent (exogenous or non-observable) factors are theoretical constructs that cannot be observed directly. Therefore, latent factors include observable/manifest (endogenous or dependent) variables. In our model, the four dimensions are latent factors (Cognitive learning process, Enjoyment of the game, Flow during the game, Learning outcome) emerged based on the 22 observed variables (C1-L7) (see Table 4
With SEM, we performed a path analysis to (1) understand more precisely the correlation patterns among factors in our model, and (2) to show how much of the variation in the learning outcome of the intervention with like2be can be explained by the latent factors.
Based on our CFA, we regarded clp (cognitive learning process), ejm (enjoyment of the game), flw (flow during the game), and loc (self-perceived learning outcome) as latent factors. To analyze to what extent the four latent factors influenced the increase in knowledge about occupations through playing the serious game like2be, we defined knowledge of occupations measured in the post-test as the outcome variable. To analyze to what extent the four latent factors influenced the increase in knowledge about occupations due to playing like2be, we defined the knowledge about all occupations from the serious game measured in the posttest as the outcome variable kaj (knowledge about jobs).
To determine the direction and strength of associations among the latent factors, we performed Pearson’s correlation. The correlation matrix (see Table 7
) shows positive high and statistically significant correlations within the latent factors.
First, a multiple linear regression showed that the overall final adjusted model explained 17% of the variance in the outcome variable (F
(4410) = 20.87, p
< 0.001). Second, we analyzed the impact of the latent factors using SEM. The initial model fitting indices by using the Satorra–Bentler adjustments for the SEM were as follows: χ2 (df) = 1.284 (221), CFI = 0.960, RMSEA (p
-value) = 0.043 (p
= 0.978), SRMR = 0.044. Overall, the SEM model fitted the data well. In this context, the factor clp explained 14% of the outcome variable kaj (β = 0.43, EST = 13.97, SE = 6.11, z
= 2.29). Additionally, the influence of the latent factor on the outcome variable was statistically significant (p
= 0.022). However, the three remaining latent factors ejm (β = 0.12, EST = 1.78, SE = 1.94, z
= 0.91, p
= 0.361), flw (β = −0.23, EST = −3.91, SE = 3.62, z
= −1.08, p
= 0.280), and loc (β = 0.10, EST = 1.93, SE = 2.91, z
= 0.66, p
= 0.507) had lower factor loadings than clp, were not statistically significant, and therefore showed no significant effect on the outcome variable (see Figure 2
In this study, we emphasized the effectiveness of serious gaming for purposes of vocational orientation. In particular, we examined the extent to which cognitive, affective, and motivational factors have an impact on measured learning outcomes in the context of an intervention with the serious game like2be.
First, we developed a scale to evaluate the effectiveness of like2be
. The basic model (Effectiveness Scale for like2be
) included nine factors (see Table 1
). Despite adopting all items from reliable and valid scales (Iten and Pekto [29
]; Fu and colleagues [63
]), our basic model was found to be inadequate in the course of the CFA. The fit indices did not indicate a good model fit. Consequently, we optimized the model. For this, we reduced our model to four dimensions based on current research. The first dimension included items in relation to measuring the extent of stimulation of cognitive processes (1st factor), which were considered to be particularly effective for learning [51
]. The second dimension referred to items that measured the extent of enjoyment of the game (2nd factor), which were considered to be of great importance in terms of learning success [29
]. The third dimension contained items measuring the flow experience (3rd factor), as state of immersion is considered to be a very important component in the effectiveness of serious games [63
]. Finally, the fourth dimension included items measuring self-perceived learning outcome (4th factor), which motivates gamers to continue playing [63
]. After reducing the basic model to four dimensions, we excluded a few items because they were either indistinguishable from other items or severely disrupted the model. Despite the fact that we could show with a CFA that our final adjusted model fits the data well, we should note that it deviates considerably from the factor models in past studies. One possible explanation for this could be that the serious games in the two studies we referred to were not comparable to the serious game like2be
in terms of gameplay, subject matter, content, controls, etc. However, our final adjusted model is reliable and valid and is suitable for further evaluations of the effectiveness of serious games, especially in the field of vocational orientation.
Second, by applying regression analysis, we investigated the overall impact of the four latent factors on increase in knowledge about occupations. Our results indicate a statistically significant impact explaining 17% of variance. Despite the proportion of explained variance being rather small, cognitive, affective, and motivational factors turned out to positively influence knowledge acquisition and contribute to effective learning with serious games.
Third, we conducted a SEM to analyze the extent to which the four latent factors influence the measured increase in knowledge about occupations. Regarding H1
, the results showed that stimulation of cognitive processes while playing like2be
had a positive and statistically significant effect on increased knowledge about occupations. Therefore, we accept H1
, that serious gaming stimulates cognitive processes and thus supports the expansion of knowledge about occupations. Regarding the importance of problem-based learning in gameplay [51
], and game challenge, as well as the prospect of winning [52
], our data analysis showed that the challenge of achieving the game objectives, the game strategies necessary to do so, and also prior knowledge, all had an impact on the expansion of knowledge. According to Sailer and Homner [53
], collaboration and competition are important for cognitive stimulation. Although, like2be
offers few possibilities for collaborative play, it includes competitive aspects because of the game time limit. Despite not evaluating effectiveness on attention capacity [54
], short-term and visual memory performance [55
], we conclude that the serious game like2be
stimulates players’ cognition with its competitive, challenging, and problem-oriented gameplay during the game and therefore has a positive impact on knowledge enhancement.
Furthermore, the data analysis indicated that how much players enjoy playing like2be
has no significant effect on increasing knowledge about occupations. Thus, we reject H2
, that the serious game like2be
is highly enjoyable and supports the expansion of knowledge about occupations. In accordance with Iten and Petko [29
], we did not find a clear connection between enjoyment and learning outcome (i.e., increased knowledge) either. In line with Wouters and colleagues [50
], we assume that like2be
was not as entertaining as other commercial computer games and was therefore less motivating. Additionally, the lack of an exciting narrative or story in like2be
with high exploration, challenge, or success aspects may have led to a lower impact of this serious game on learning outcomes [62
Moreover, the results of SEM indicated that the flow experience while playing like2be
had no significant effect on improving knowledge of occupations. Accordingly, we reject H3
, that the serious game like2be
stimulates the flow experience. Although, the serious game like2be
provided intellectual competition in an entertaining way [22
] and aspects such as comprehensible game goals, rules and gameplay, plus feasible challenges were given, it neither led to a GameFlow state [63
] nor to a situation of complete absorption in the gaming activity mentioned by Csíkszentmihályi [64
]. A possible explanation is offered by the study results relating to enjoyment. Thus, we assume that players did not experience a particularly high level of enjoyment while playing like2be
for several reasons.
In terms of H4
, the data analysis showed that the extent of subjectively experienced benefits through like2be
had no significant effect on increasing knowledge of occupations. Therefore, we reject H4
, that the serious game like2be
leads to high level of self-perceived benefit. In this context, like2be
did not contain in-game scaffolding [68
] or did it provide players with a way to ask for feedback [51
]. In this respect, like2be
does not include in-game opportunities for additional reinforcement of the content, which would consolidate learning and help players realize that they are learning successfully through serious gaming. However, there is additional teaching material to deepen learning, following Wouters and colleagues [50
], who mention that serious games are most effective for learning when supplemented with additional teaching materials. Furthermore, a set of analog activities for the serious game like2be
already exists and includes reflection, modeling, collaboration, or personalization activities designed to support learners and help them continue to learn successfully [50
]. The materials have even been favorably evaluated but were unfortunately not part of the current investigation.
Recent studies have shown that serious games can be used effectively for various educational purposes, in particular to promote the acquisition of specific technical knowledge, but also to enhance factors that can promote effective learning. In this study, we focused on the impact of the serious game like2be on factors conducive to learning, such as increasing cognitive performance and enjoyment of the game, stimulation of a flow experience and self-perceived benefits through serious gaming.
As a result of our data analysis, we conclude that like2be
stimulates cognitive processes in players, which consequently has a positive effect on their expansion of knowledge about occupations. However, we were not able to demonstrate that the impact on enjoyment, flow experience, and self-perceived benefit through like2be
promoted the expansion of knowledge about occupations. Indeed, these findings are in accordance with or can be explained by previous research. For example, enjoyment of a serious game was found to be very important for learning outcome [63
], but to provide enjoyment, serious games must include an exciting narrative or story [62
]. Additionally serious games must be as highly enjoyable as commercial computer games in order to enhance motivation and thus learning outcomes [50
]. Since the fun factor of serious games is not considered competitive with commercial computer games for several reasons [50
], it is not surprising that in their intervention study Iten and Pekto [29
] found no impact of enjoyment on measured learning outcomes. Considering that enjoyment of the game is a relevant indicator of immersion in an activity [63
], we assume that the perceived level of enjoyment while playing like2be
was too low and therefore failed to promote a Flow experience. Further, studies have shown that when players had an in-game opportunity to request scaffolding [68
] or feedback [51
], learning processes with serious games were effectively enhanced. Due to the fact that like2be
does not provide in-game scaffolding or feedback, it was not surprising that the data showed no impact on expansion of knowledge about occupations. Furthermore, Wouters and colleagues [50
] have shown that additional instructional materials such as reflection, modeling, collaboration, or personalization [50
] can enhance the impact of serious gaming. Although, such additional teaching materials have been developed and are available for like2be
, their impact was not considered in the context of this study.
On the positive side, our scale (final adjusted model, see Table 6
) can be used for future evaluations of the effectiveness of serious games, as it was found to be reliable and valid after testing with confirmatory factor analysis (CFA).
In conclusion, the serious game like2be stimulates cognitive processes through its competitive, challenging, and problem-oriented gameplay, consequently promoting effective learning. Thus, it represents an effective tool and should be considered for use in career choice classes.
For the future, we recommend intensifying research in the field of the effectiveness of serious games, especially in the field of vocational orientation. Regarding like2be, the effect of the additional teaching material on cognitive, affective and motivational factors conducive to learning should be analyzed. Furthermore, our data analysis shows that the game can be optimized in terms of enjoyment, flow experience, or self-perceived benefits. In this respect, like2be should be evaluated with a view to identifying opportunities to add more fun, flow experience indicators, or a scaffolding component to the gameplay. With regard to an optimization of like2be as well as the development of other serious games, we therefore recommend a stronger collaboration between game developers, educational experts and the target groups of players. We consider educational experts (e.g., teachers or educational scientists) as ideal contacts for developing specific scaffolding as well as an adequate and exciting narrative. Nevertheless, we suggest asking the target groups pf players (e.g., students) about which aspects make computer games exciting and appealing for them. The findings can be incorporated into the development of new or the optimization of existing serious games and thus generate a high degree of enjoyment and flow experience.