Techniques to Motivate Learner Improvement in Game-Based Assessment

Abstract

Learner motivation to self-improve is a crucial effectiveness factor in all modes and settings of learning. Game-based learning was long used for attracting and maintaining students’ interest especially in small ages, deploying means such as scoring, timing, scores of peers (i.e., hall of fame), etc. These techniques can provide recognition for high-scoring players, while also developing a sense of safe “distance” in the impersonal electronic environment for low-scoring players. In addition, constructive feedback on mistakes a player makes can contribute to avoiding similar mistakes in the future, thus achieving better performance in the game, while constructing valuable new knowledge when a knowledge gap is detected. This paper investigates an integrated approach to designing, implementing, and using an adaptive game for assessing and gradually improving multiplication skills. Student motivation is fostered by incorporating the Open Learner Model approach, which exposes part of the underlying user model to the students in a graphically simplified manner that is easily perceivable and offers a clear picture of student performance. In addition, the Open Learner Model is expanded with visualizations of social comparison information, where students can access the progress of anonymous peers and summative class scores for improving self-reflection and fostering self-regulated learning. This paper also presents the feedback received by the preliminary testing of the game and discusses the effect of assessing multiplication skills of primary school pupils using the adaptive game-based approach on increasing pupil motivation to self-improve.

1. Introduction

Elementary and secondary education is basically targeted at children’s development of “basic academic and social skills and noncognitive attributes (e.g., perseverance, dependability, and consistency) necessary for productive citizenship and entry into the workforce, a training program, or college” [1] (p. 1511). Although a wide range of educational outcomes is desired, policymakers center on the learning of specific subject material. When lack of learning is observed, it can trivially be attributed either to learners themselves or to problems related to other contributing factors to students’ learning (e.g., non-optimal teachers, administrators, parents, and peers). In the first case, learners may lack motivation to learn or may highly underestimate the long-term outcomes of the academic effort and, therefore, alternatively use their time. Thus, it is quite significant for learners to be supported in the learning process educationally and socially, so as to keep striving to achieve success.

Research on motivation is based on theories that share, in a high grade, the same or similar concepts. For example, behavior-based theories of learning [2,3], which conceptualized motivation in terms of habits, drives, incentives, and reinforcement schedules, were popular through the mid-20th century. According to these approaches, during the learning process, learners remained passive and, thus, research emphasized people’s differences like cognitive abilities, drive for achievement, etc. As these differences were claimed to be stable and prearranged, they were believed to determine learners’ reactions toward the learning environment’s aspects (e.g., method of instruction and incentives), as well as learners’ motivation and performance.

Although these factors remain significant, current researchers support the belief that learners actively participate in the learning process and encounter, with great respect and attention, the ways a learner understands the learning environment and prefers to engage with it. As a result, cognitive theories focus on the modes where a learner sets goals for learning and achievement and, furthermore, the ways one records and observes their own progress relating to those goals. Motivation is claimed to be an emerging phenomenon, as it dynamically changes over time and alters according to one’s learning and more general experiences. Hidi and Renninger [4] believe that the learning environment’s parts can both cause and maintain a learner’s curiosity and interest in order to promote motivation and learning. A critical role in motivation is played by an individual’s mindset. According to Dweck [5], mindset is defined as the set of assumptions, values, and beliefs about the world and oneself that affect the way one perceives, interprets, and acts upon one’s environment. Mindsets change through time as a result of learning experiences and cultural influences. Research on mindsets explored patterns in goal interpretation and making choices related to direct attention and effort. Blackwell et al. [6] support the opinion that students can possibly change their self-attributions in order to adopt a growth mindset and, thus, achieve improved academic performance.

2. Motivation in Learning

Motivation “forces acting either on or within a person to initiate behavior” [7]; it also “refers to a desire, need, or drive that contributes to and explains behavioral changes” [8]. “Motivation is a condition that activates and sustains behavior toward a goal” [9] (p. 109). It plays a very central role in learning and achievement on many levels of everyone’s life, as well as in both formal and informal learning scenarios. Thus, according to Pintrich [10], when pupils are motivated, then they are engaged, persist longer, have better learning outcomes, and perform better than non-motivated peers on standardized achievement tests. Brown defines motivation as “an inner drive, impulse, emotion, or desire that moves one to a particular action” [11] (p. 114). Therefore, a motivated learner is the learner “who wants to achieve a goal and who is willing to invest time and effort in reaching that goal” [12] (p. 1187). Ur [13] claims that, among learners, the most successful are those that are characterized by the following specific capacities that are attached to motivation:

• Positive task orientation. The learner feels ready to deal with tasks and challenges and is self-confident about succeeding.
• Ego-involvement. The learner believes in the importance of succeeding in learning, so as to preserve and boost one’s good self-image.
• Need for achievement. The learner has an urge to succeed and achieve personal goals, as well as to overcome difficulties.
• High aspirations. The learner sets high goals and believes in one’s success.
• Goal orientation. The learner has a clear idea about the learning goals or certain learning activities and dedicates consciously the required powers in order to meet them successfully.
• Perseverance. The learner is persistent in maintaining high standards in learning and is not discouraged by any kind of difficulties.
• Tolerance of ambiguity. The learner does not feel desperate in case of misunderstanding or not understanding; on the contrary, they feel secure that these setbacks will be overcome sooner or later.

3. Learning Styles and Motivation Theories

3.1. Learning Styles

According to Bain [14] (pp. 40–41), there are three categories of learners shaping different learning styles:

• Deep learners, who enjoy the challenge of dealing with a demanding and difficult subject. These learners are intrinsically motivated.
• Strategic learners, who are mainly motivated by rewards. These learners cope effectively with the competition and the scenario of outmatching peers. Despite their high school performance, their engagement will be true and original, if a clear reward is promised. These are characterized by the “bulimic” behavior in their learning as it is short-lasting until the end of a test or exam and then they rapidly “vomit” the corresponding material.
• Surface learners, who are usually motivated by the urge of not failing. Their effort is the minimum required in order to pass a course or an exam and typically their learning is more superficial, as they consider deep learning as inherently risky and prone to failure.

Despite the fact that motivation is a key component to instruction and learning, there does not seem to exist one standard definition for the hypothetical construct of motivation. Initially, research about motivation was connected with primitive drives and needs [15]. In a more modern approach, Moshinskie claimed that motivation refers to “the attention and effort required to complete a learning task and then apply the new material to the work site” [16] (p. 34). Between these two approaches stands Bandura [17], who identified three different forms of motivation that lead to the development of corresponding theories: attribution theory, expectancy-value theory, and goal theory [18].

3.2. Attribution Theory

Humans are motivated to attribute causes to their actions and behaviors [19]. According to social psychology, attribution is the process via which individuals explain the causes of behavior and events. Models to explain this process are called attribution theory [20]. Attribution theory concerns the ways a learner perceives successes and failures. The success or failure on an assignment may be ascribed by a learner to oneself or to external reasons. The causes of success or failure can be fixed and/or controllable over time. Attribution theory reveals at least one assumption regarding motivation. Learners should be supported to assign their learning outcomes with a controllable and unfixed form of effort; otherwise, they will not be motivated to participate in a learning experience if they believe that change (and, thus, success) is impossible [18].

3.3. Expectancy-Value Theory

The general idea of expectancy-value theory is that there are expectations, as well as values or beliefs, which affect subsequent behavior. Consequently, learners expect certain outcomes from behaviors and, thus, when learning outcomes are assigned with high value, then it is more possible for the learner to demonstrate the necessary behavior. Expectancy-value theory depends strongly on the conceptions of what the learner can achieve [18].

3.4. Goal Theory

According to goal theory, by establishing goals to be obtained, behavior is motivated. The goals that a learner sets can be either learning or performance goals. “Performance goals foster the implicit belief that intelligence is fixed, while learning goals are associated with the belief that intelligence is malleable and can be developed” [21] (p. 309). Performance goals are those that concentrate on the evaluation of one’s competence in a specific area. For example, the learner wishes to reach a specific performance level compared to some preset scoring standard. Learning goals are those that concentrate on the learner developing new skills, knowledge, or attitudes and not targeting on some performance level or judgment. Goals can also be distinguished as proximal and distal. Proximal goals are those that can be achieved in a reasonably short time period, whereas distal goals are those that will be reached after a long period of time. Typically, proximal goals are associated with preserving motivation and, thus, are desired, although it is well known that setting goals is not enough to maintain motivation [18].

Learners cannot measure their progress toward success without some measure of progress toward the goal. Despite the fact that a single standard definition for the construct of motivation does not exist, all three theories have links to self-efficacy involved with motivation. “Perceived self-efficacy refers to the beliefs in one’s capabilities to organize and execute the courses of action required to produce given attainments” [17] (p. 3). In attribution theory, when a learner ascribes success to ability, it leads to higher beliefs of personal efficacy, which followingly predicts the learner’s future results. According to expectancy-value theory, once a learner succeeds in achieving the learning of a particular task, self-efficacy is increased and, hence, the possibility of attempting the task in the future is heightened. Goal theory and self-efficacy are strongly connected. For example, a learner’s perceived abilities will impose the kind of goals one sets. Additionally, as a learner makes the effort to achieve a goal, the successes and failures that are met will contribute to their own beliefs of self-efficacy. Closely connected to all these theories are the concepts of intrinsic and extrinsic motivation [18], which are discussed next.

4. Motivation Categories

Motivators provide some sort of incentive for completing a task. There are two categories of motivation based on the nature of the motivator: intrinsic deriving from internal factors and extrinsic deriving from external factors [8]. According to the Incentive Theory, human behavior is originally extrinsically motivated; people are, to a higher degree, motivated to perform activities if they are promised of receiving a reward, rather than due to the enjoyment they will feel during an activity; therefore, most incentives derive from extrinsic sources.

The theory of student learning strongly includes motivation (both intrinsic and extrinsic). The existence of many incentives can support learners to strive harder. Deci et al. [22] claim that the way learners respond to motives or incentives is based on several key factors, including the student’s own goals and level of intrinsic motivation, which in fact determines the significance of the task for them, and their understood possibility of success. On the other hand, extrinsic rewards are believed to be helpful in motivating learners to be engaged to unpleasant work, but these kinds of rewards were found to be ineffective and even harmful in some circumstances [1].

4.1. Intrinsic Motivation

Intrinsic motivation refers to a learner’s internal desire to perform a task and only be rewarded with personal satisfaction and enjoyment [18]. Intrinsic motivation derives from the learners and their attitudes toward the topic, their learning goals and aims, their emotions, and their ambitions [12]; according to Deci, “intrinsically motivated activities are ones for which there is not apparent reward except the activity itself. People seem to engage in the activities for their own sake and because they lead to an extrinsic reward. Intrinsically motivated behaviors are aimed at bringing about certain internally rewarding consequences, namely, feelings of competence and self-determination” [23] (p. 23).

Motivators like “fascination with the subject, a sense of its relevance to life and the world, a sense of accomplishment in mastering it, and a sense of calling to it” [24] (p. 163) are intrinsic. Intrinsic motivation can be long-lasting and self-sustaining. When trying to cultivate intrinsic motivation, in fact, student learning is promoted, while the subject is the center of such efforts and not rewards or penalties. Although behavior will be influenced by adapting intrinsic motivation, this will be done slowly and, furthermore, a personalized and time-consuming preparation is necessary. As each learner is different, different approaches are necessary to motivate everyone. It is a good idea to connect every learner’s interests with the subject matter and, to achieve this, it is necessary to get to know every learner [24].

Middleton et al. [25] (pp. 255–257) proposed a model of academic intrinsic motivation; initially, when given the opportunity to engage in a learning activity, a learner decides whether the activity is interesting. If so, the learner engages in the activity, while, if it is not, the learner assesses the activity on two factors: the stimulation it offers, and the personal control it promises. If the activity is evaluated as stimulating and controllable by the learner, then the learner trial names the activity interesting and engages in it. If either condition is not fulfilled, then the learner disengages from the activity, except an extrinsic motivator appears and persuades the learner to go on. If the activity is repeatedly deemed stimulating and controllable, it leads the learner to possibly assess the activity as interesting and, thus, in the future they will be presumably engaged with the activity. In the opposite case, the activity will be removed from the learner’s mental list of interesting activities.

4.2. Extrinsic Motivation

In the case where a learner is motivated by rewards and incentives external to personal interest and satisfaction, then these factors are termed extrinsic motivators [18]. External motivation is produced by external incentives like money, prize, grades, positive feedback [11], the learner’s purpose to satisfy parents, and the desire to attain high assessment in an external exam and to be the best among peers [13]. Motivators like “parental expectations, expectations of other trusted role models, earning potential of a course of study, and grades” [24] (p. 163) are extrinsic. Extrinsic motivators lead more easily to behavior changes and usually do not demand much effort and preparation. Furthermore, it is often not necessary to have a deep knowledge of individual students, in order to apply extrinsic motivators. However, there are some drawbacks, as extrinsic motivators can often distract students from learning the subject at hand. In addition, it is critical to form appropriate rewards and penalties for student behaviors and to differentiate them for each learning stage, as it is common for students to lose impulse, when rewards or penalties stop being valid.

Ur [13] highlighted the following conditions that influence extrinsic motivation:

• Success and its rewards. A learner who was successful in previous activities is expected to be more actively engaged in future tasks, due to stronger self-confidence in success and the cultivation of consciousness to recognize success.
• Failure and its penalties. Learners should keep in mind that a failure means not only unsatisfactory progress, but it is also occasionally a normal part in any learning experience. Thus, not only should the learner not be embarrassed, but they should gain from an experience like that and use it creatively, to move safer toward a next success.
• Authoritative demands. Especially younger learners claim teachers to be the basic authority, whose demands and wishes should be obeyed and followed.
• Tests. Tests can be quite motivating for learners because, through testing, a specific threshold is set for learners to prove their acquired skills and knowledge.
• Competition. Through a competition, learners are usually expected to maximize their effort. Group competitions outmatch individual ones, as they are more enjoyable and less stressful [12].

Whereas intrinsic motivation concerns the performance of an action unattached to enjoyment or interest, extrinsic motivation is produced by externally or socially created causes to perform an action. Extrinsic motivators (money or other rewards) can produce extrinsic motivation as they cause urge for the result the activity; they do not provoke desire to engage in the activity for the sheer activity. In these circumstances, motivation is firmly established in the environment rather than within the learners themselves. On the contrary, intrinsic motivation exists within the learner and can be exploited and intensified by environments that promote autonomy and competence of the learner. Intrinsic motivation exists under people’s natural tendencies to look for innovation challenge, to learn, and to progress [26]. In contrast to extrinsic motivation, intrinsic motivation is attached to creativity and vitality [27].

4.3. Other Motivation Categorization

According to Brown [11], motivation can also be distinguished into three other categories: global, situational, and task-oriented. Global motivation refers to the general orientation a learner has toward the goals of the learning task. Situational motivation is connected to the situation where the learning takes place (classroom learning, naturalistic learning, etc.). Task motivation refers to the motivation of a learner to implement a specific activity [12]. A learning activity requires most of the three types of motivation. For example, when learning a foreign language, a learner can have high global motivation, but low task motivation for implementing the particular activity [11].

5. Motivation in Gamification

A plethora of popular and academic publications on serious games and gamification were observed during the past recent years. Although terms “gamification” and “serious games” suggest different notions (serious games are game environments, whereas gamification uses game mechanics and techniques in non-game environments), they share many similar features and they are often used interchangeably [28]. Games’ basic purpose is entertainment; however, through their universal applicability, they support a wealth of other aspects of everyday life: training and knowledge sharing in domains as defense, education, scientific exploration, healthcare, emergency management, city planning, engineering, religion, government and non-governmental organizations (NGOs), business, marketing, communication, and politics [29,30,31]. These games are defined as “serious games”, whose main purpose is to train, investigate, or advertise [29,30,31]. In coherence to serious games, gamification is the application of game elements for purposes beyond mere entertainment [32,33]. Serious games and gamification are correlated as they both try to reclaim games’ characteristics in order to manage something beyond playfulness. On one hand, serious games provide an entertaining way to find solutions for real-world problems, while, at the same time, with gamification, they can both be smart ways to promote a business or product. Gamification utilizes the motivational power of techniques, mechanics, and dynamics of games, in a way that promotes participation, persistence, and achievements. Moreover, the idea of using game mechanics and dynamics to support participation and engagement is well worth being examined. Gamification aims to create a sense of playfulness with the application of game-design elements and game principles in non-game environments, so that participants feel enjoyment [34,35,36]. Gamification combines both intrinsic and extrinsic motivation, as, on one hand, there is a use of extrinsic rewards to support engagement, while, on the other hand, there is an effort to promote intrinsic motivation characteristics like achieving mastery, autonomy, and sense of belonging [29]. Among the typical game design elements, those with the strongest effect on extrinsic motivation are points, badges, leaderboards, performance graphs, meaningful stories, avatars, and teammates [37].

• Points are basic elements of a multitude of games and gamified applications [38]. They are typically rewarded for the successful actualization of defined activities within the gamified environment, while they also numerically represent a player’s progress [39,40]. Through points, players’ in-game behavior can be quantified and, therefore, they provide continuous and immediate feedback or even a reward [41].
• Badges optically represent achievements [39], and they can be gained and gathered inside the gamification environment. They visibly demonstrate players’ accomplishments of level or goals and their worth [42,43]. Like points, badges also provide feedback as they indicate players’ performance [44]. Although collecting is not compulsory, they can influence players’ behavior, as they direct players to choose certain routes and challenges, so as to earn the associated badges [37,45]. Furthermore, badge ownership places players in a specific group and, thus, badges can exert social influences on players and co-players [42,46].
• Leaderboards enlist names and current scores of the leading competitors according to their relative success, measuring them against a defined success criterion [47]. Thus, leaderboards record the best performance in a certain activity [48] and act as indicators that compare players’ progress. However, the motivational role of leaderboards is controversial [49], because they are regarded as effective motivators for players in positions too close to the upper rank, and as demotivators if players are ranked among the last positions of the leaderboard [37]. Competition occurring via leaderboards can increase the player’s engagement as it is based on social pressure and can, therefore, support participation and learning [50].
• Performance graphs offer information about the players’ performance in comparison with their previous performance during a game [41]. Unlike the social contest orientation of leaderboards, performance graphs have an individual-through-time contest orientation. Players’ performance over a certain period is graphically presented, and there is a focus on improvements. According to the motivation theory, this promotes mastery orientation, which is notably effective for learning [37].
• Meaningful stories are not related to the player’s performance. A gamified application can be incorporated in a narrative framework, which contextualizes activities and characters in the game, and it obtains an additional meaning beyond the mere quest for points and achievements. A story can be either just a game’s title or even complex storylines that are typical of contemporary role-playing video games [51]. Narrative contexts can function as analogies of real-world settings and, if the story correlates with players’ personal interests, it can inspire and motivate them [52]. Therefore, stories play a significant role in gamification applications, as they can change the meaning of real-world activities by adding a narrative “coat” [37].
• Avatars are visual representations of players within the game or gamification environment [49]. Usually, a player can choose or even create them [51]. Avatars can have a quite simple design as a mere pictogram or they can be complexly animated, three-dimensional representations. Their basic role is to distinguish players from other human or computer-controlled avatars [40].
• Teammates can be other real players or virtual non-player characters (NPCs), and they aim to prompt conflict, competition, or cooperation [51]. Cooperation is specially supported by introducing teams, i.e., by creating fixed groups of players that collaborate toward a shared objective [49].

All these game design elements share strong motivational influence, which becomes clear through a deep study of self-determination theory. According to this theory, behavior is strongly determined by three universal, innate, psychological needs:

• Autonomy refers to the strong desire one has to control their own life [9], as well as to the psychological freedom to achieve a certain task [37,53]. Psychological freedom refers to the sense of making decisions according to one’s own values and interests [54,55]; therefore, in autonomy, one can behave without external pressure or enforcement [53].
• Competence refers to the strong desire to experience mastery [9], as well as to develop a sense of efficiency and success in the interaction with the environment [44,56]. It is claimed that humans have the urge to feel competent when intentionally affecting the environment they interact with [37].
• Social relatedness refers to the strong desire to interact with, be connected to, and care for others [9]. It depicts the urge of the individual for coherent integration with the social environment [37,57].

Researchers linked this theory to people’s intrinsic motivation to learn [58,59,60]. As mentioned above, intrinsic motivation is the experience of wanting to engage in an activity for its own sake because the activity is interesting and enjoyable or helps to achieve goals one chooses. According to self-determination theory [58,59,60], if a high degree of autonomy exists, then learners are intrinsically motivated to learn and engage in a problem or task willingly; furthermore, these learners comprehend that the challenges of an activity are within their abilities [9].

In an effort to correlate these three intrinsic psychological needs with game-based elements, it is easy to observe that, as collected points are immediately influenced by a player’s actions, they offer a quantified view of a player’s progress and, therefore, the need for competence is addressed [37]. Performance graphs represent not only players’ performance, but also the level of competencies and, thus, the need for competence is satisfied. The need for competence is also met in badges, as badges are directly connected to a player’s progress and in leaderboards, as these visually rank players’ performances. As the need for autonomy can be expressed in two forms (experience of decision freedom and experience of task meaningfulness), it can be claimed that avatars satisfy this need due to the freedom of choice they offer to players [61]. On the other hand, meaningful stories satisfy the second aspect of the need for autonomy, as, through stories, players experience their choices in a meaningful and engaging manner [37,44].

6. The Multiplication Game

It is beyond any doubt that learning the multiplication tables is a significant topic in the curriculum of mathematics education. Succeeding in plain multiplication facts facilitates the calculation of solutions to more complex problems. In the educational field, several games and programs were developed to make practicing multiplication facts easier and more enjoyable; nevertheless, the memorization of these facts can be a laborious and prolonged task for pupils in primary education [62,63,64,65,66,67]. In particular, some multiplication facts are even more difficult to memorize and, therefore, both children and adults make more mistakes and need more time to learn [68].

The Multiplication Game (MG) is a web-based practice and progress monitoring application pursuing scientific and educational aims. This application, which is currently in its fourth stage of development, started as an adaptive mobile application (version 1-v.1) [69] and was revised as a desktop application with incorporated Open Learner Model (OLM) elements (version 2-v.2) [70]. An OLM offers to students (and other stakeholders of the learning process) easily perceivable access to personal progress information [71]. Next, the MG was expanded by adding social characteristics to the OLM (social OLM or OSLM), in the sense that the learner model is open not only to the learner, but also to teachers and peers (version 3-v.3) [72]. Studies [73,74,75] consistently support the assumption that access to the models of peer learners results in students covering more topics in the system and reaching higher success rates in self-assessment problems. The MG is basically an educational tool whose primary target is to offers pupils non-traditional ways of practicing multiplication facts according to their learning in a more pleasant and fun way, as fun introduces satisfaction to children [76]. It also provides teachers insight into their pupils’ specific mathematical skills. It is among the MG’s aims to discharge teachers from time-consuming correction of pupils’ schoolwork by providing feedback on individual pupil’s learning outcomes and analytical information on their progress. Furthermore, the MG provides researchers with detailed high-frequency data on the processes of arithmetic development. Hence, it supports research on pupils’ mathematical abilities.

The game consists of four basic levels. In each level, multiplication practice is offered through different types of questions: right/wrong choice, multiple choice, one-to-one matching, and gap filling. In the first level, the user is given multiplication facts in a sequence and has to decide whether the given answer is right or wrong (Figure 1). In the second level, the user has to select the right answer to the given multiplication fact among four choices (Figure 2). In the third level, the player has to match four different multiplication facts with four different answers (Figure 3). In the last level, the player has to answer the given multiplication without being given any helping clues (Figure 4). The questions in all four levels cover the curriculum in primary education (second to sixth grade). Each session consists of a specific number of questions depending on the number initially selected by the participant. MG uses an adaptive testing method to calculate pupils’ weaknesses and, subsequently, exercises are automatically adjusted to the certain user’s needs.

From a motivational point of view, mastering the multiplication table is a very practical skill that pupils can use in their everyday life, and this acts as a strong long-term motive for them. On a short-term basis, lower-grade pupils are often asked multiplication table quizzes at school, while higher-grade ones are asked to use these skills in many advanced topics, where multiplication is considered as prerequisite knowledge. Supporting these two levels of short- and long-term motivation for pupils to learn, the new edition of the MG was designed with many game-based elements, further fostering learner motivation. More specifically, pupils gain points in the form of coins, displayed at the top of the game interface, by answering question-items correctly. To increase the challenge for the pupils of the higher two grades, the game can be set to also apply negative scoring by subtracting coins in the case of wrong answers, in combination with player timing. Because of the adaptive nature of the test, every student is supported to progress and, by focusing on identified weaknesses, each pupil is helped to overcome them. Therefore, the number of coins won reflects pupil multiplication skills and a gradual increase indicates the achieved progress.

As personalization and supported customization options are strong evidence of intrinsic motivation [77], it is significant that the MG already strongly supports personalization through its OLM features and, furthermore, more clues are developed. Specifically, in the very beginning of playing, after user authentication, theme selection options, as well as a variety of avatar icons related to the selected theme, are offered. In addition, the feature of meaningful stories is supported at a first level by the selected theme, which defines a thematic framework for the player (e.g., sports, underwater, pirates, dragons), and at a second level, by a correlated cartoon character acting as an NPC who gives to the player information about the coming level (and optionally about individual progress) in the form of cartoon bubbles. Overall, since the game addresses schoolchildren, it is crucial to offer pleasant graphics, bright colors, related sound effects, and animation in order to optically allure learners and, thus, engage their participation.

Another way in which pupils are motivated to continue playing in the MG is that, after the completion of each level, the learner has the choice to visually be informed about previous level achievements (Figure 5) though performance graphs [70]. Studies [78] showed that people are twice as likely to achieve a goal if they are given a sense of advancement toward it. Although, until the third version of MG, plain visualization was offered in the form of smiley faces, which were considered to be more compatible to the ages of seven and eight-year-olds [79], the current MG version will be enriched with more complex forms of performance graphs. More specifically, for progress visualization, bar charts and skill meters will be offered to learners, and they will also be allowed to decide which graph fits them best. In MG v.3 [72], a strong social parameter was highlighted. Remarkably, the social aspect is important in games [80]. Competition, social interaction, or cooperation may motivate player behavior [81,82,83]. Partial access to one’s model can also include peers for comparison reasons, and that was fulfilled through leaderboards in the form of a hall of fame, where the three pupils were ranked according to their scores per multiplication table number.

Motivation supported via coins, leaderboards, badges, and levels fosters the need for status, recognition, and prestige, as well as strengthen competence and mastery. The promotion of the top three ranking pupils offers recognition to high-scoring players, while allowing a sense of safe “distance” for low-scoring players (whose scores do not appear at all). On the other hand, as every pupil belongs to a certain group (that of his classroom), the idea of teammates exists since the competition in the MG concludes with promoting all high-ranked peers from the same classroom. According to Richter at al. [28], “a combination of a progress bar and a leaderboard is likely to generate excitement, commitment, a will to finish a gamified activity in a successful manner, and even desire to repeat the experience” (p. 38). Additional MG features that promote motivation include providing help in the form of hints and displaying graphical explanatory feedback correctly depicting the wrongly answered multiplication fact for commonly observed mistakes [84]. Motivation is also enhanced by timing players that can be available only for the case of older or highly skilled pupils, in order to provoke their interest and maintain their engagement. The idea is to reward fast players with extra points.

Certain crucial points were taken into consideration when designing the internal mechanism of the MG. For instance, displayed answers to multiplication facts are not generated randomly but are intentionally selected to be informative distracters [85]; thus, a correct answer indicates genuine learning. Furthermore, as already mentioned, the MG incorporates an adaptive algorithm that diagnoses, after each level completion, the learner’s main weakness and, by focusing on this particular number in the next level, the game helps pupils overcome their difficulty.

Preliminary tests were conducted on both v.2 [70] and v.3 [72] of the MG. A questionnaire was provided about the usability and usefulness of MG, as well as the OLM and OSLM available features, where particular questions with a focus on the motivational value of the OLM and OSLM visualizations are offered. The feedback received by both pupils and teachers was quite encouraging and supportive of the positive effect on increasing pupil motivation to self-improve when assessing multiplication skills using the adaptive game-based approach. More specifically, MG v.2 [70] was tested with 36 pupils (17 girls and 19 boys) of a primary school. Pupils answered a Likert-type scale questionnaire from 1 (definitely not) to 5 (surely yes) about the usability and usefulness of MG and the available OLM elements. Pupils after playing the MG at home from three to six times were asked to fill in the questionnaire in

Table 1. Questionnaire used for assessing the Multiplication Game (MG) v.2.

No.	Question
Q1	I enjoyed playing the Multiplication Game (MG)
Q2	I liked the interface of MG.
Q3	The given instructions were enough to understand how to play MG.
Q4	I found it useful to see my progress in MG.
Q5	Seeing my progress in MG made me realize how well I know the multiplication table.
Q6	Seeing my progress in MG motivated me to plan for specific homework.
Q7	It was useful to see my progress in each level/different type of questions.
Q8	I find the used visualization (smiley faces) to be a good idea.
Q9	I believe MG is more a game than a lesson.
Q10	I believe MG is more a lesson than a game.
Q11	I would like to play MG at my home personal computer (PC) as well.
Q12	I would like to see the scores of my peers.
Q13	I would like to see the class average score.
Q14	I believe that seeing the progress of others would motivate me to work harder.
Q15	It is important for me to see my rank among my peers.

. MG v.3 [72] was tested by 41 pupils (18 girls and 23 boys) of the third and fourth grades of a primary school. Pupils played the MG during two school teaching hours of the information and communication technology (ICT) course and then were asked by their teacher to answer the questionnaire in Figure 6 (a revised version of the questionnaire in Reference [70]) about the usability and usefulness of the MG and the available OLM and OLSM elements. A smileyometer, a discrete Likert-type scale [86], was used for the needs of the survey in v.3 (Figure 6). In both cases, the number of participating pupils was not adequate to try observing any possible differences related to students’ age. Moreover, questions also focused on the motivational value of the provided OLM visualization. In MG v.3, the teachers of the corresponding classrooms’ pupils were given also a Likert-based questionnaire in order to express their opinion about MG. These teachers had the opportunity to access and assess (through the application) their pupil’s learner models (LMs). They were offered the opportunity to select and explore the progress history of certain pupils and the whole class on average. The practical value of MG v.3 was positively assessed by teachers, who were satisfied by the provided functionalities. Being questioned about the educational role of the game, teachers claimed that the MG can indeed support their traditional instructional procedure and help pupils master multiplication skills in a pleasant and motivating way [72].

In summary (details on the assessment of both MG versions can be found in References [70] and [72]), the large majority of pupils (more than 92%) found playing the MG enjoyable, and more than 87% considered the application’s interface alluring. More than 81% believed that the provided instructions on how to play the game were adequate. Concerning the meta-cognitive role of MG, more than 75% believed in the usefulness of having access to their model for seeing their progress, while more than 65% stated that self-reflection and self-motivation are being supported. More than 75% considered important the visual depiction of their progress after completing each level and, thus, watching their achievements in different question types. More than 87% positively perceived the smiley faces as the type of visualization provided. Being questioned whether the entertaining or educational nature of application was more obvious, more than 67% felt that the entertaining sense was more dominant in the tool. More than 85% wanted to be offered the opportunity to play the game at home, which gives extra support to the assumption that pupils enjoyed interacting with the tool. Overall, 85% were interested in seeing their progress history and being informed whether they had improved or not, while 73.2% were interested in knowing the class average progress. Furthermore, 87.8% were interested in being compared with the class average progress, 63.4% supported the idea that being compared with the class average leads to self-improvement, and 80.5% enjoyed being informed about their top three classmates. The social parameter of the top three classmates would motivate 63.4% of the participants, whereas less than 25% believed that this information would have a negative impact on all pupils not included in the top three list. This finding is interesting, as providing access to the score of others seems to motivate pupils in self-directing their study. Moreover, 73.2% would like the idea of being able to access the model of a specific peer/classmate. In conclusion, questionnaire responses suggested that pupils have positive reactions toward the OLM and the OSLM approach. They enjoy interacting with it without underestimating the educational role it plays. Our findings also support the meta-cognitive role of OLM/OSLM.

The teachers of the participating classes had the opportunity to access their pupils’ learner models (LMs) and assess their progress. They were able to explore individual pupils’ progress history and see the average class progress history. All shared a positive opinion about the practical value of the application, and they were satisfied by the provided functionalities. Being questioned about the educational role of the game, teachers responded that the application can indeed support their traditional instructional procedure and help pupils master multiplication skills in a pleasant, motivating way. In addition, MG can support teachers by offering more effective pupil assessment, minimizing time and required effort.

7. Conclusions

This paper presented the design and updating of an existing web-based training/tracking system, in which primary school students can practice their multiplication facts skills while their activities are simultaneously recorded. In MG, educational and research goals are met, as it is a necessary instrument that measures pupils’ multiplication fact ability, supports different individual levels, and supports motivation by providing game-based elements.

After the completion of the development of this last version of MG, it was planned to examine the effectiveness of this instrument in real classroom conditions during the school year with the participation of many pupils per grade, in order for pupils’ performances to be more thoroughly recorded and safer conclusions to be reached. One of the MG targets is to offer a challenging web environment for children of all competency levels. By analyzing usage statistics, a conclusion could be reached about the grade of motivation children felt to play the MG and, therefore, an indication is provided of how attractive and challenging the participants found MG. It is quite likely that pupils visit the MG site mainly because their teachers told them to; however, examining if they participate and how long their participation out of the school hours is would provide interesting findings.

Another experimental idea is the case of investigating the application in two versions (with and without the OLM elements), as, through a comparison like that, interesting results would occur. These experiments will lead to a comparative analysis of OLM and non-OLM game versions on the basis of learning outcome, as well as student preference, metacognition, and motivation.

MG offers teachers the chance to access information about either individual or summative learner models in the system; therefore, they can quantify the effect of the learning process and accordingly plan subsequent teaching activities in the classroom. A large-scale testing set-up with an adequately long duration and an implementation of circles of intermediate interviews with the teachers and pupils can be used to collect both quantitative and qualitative data.