Integrating Science Tasks and Puzzles in Computer Role Playing Games

Abstract

The design of educational serious games to be used as motivational learning environments is very rewarding but also very challenging. The integration of learning with playing activities seems to be one of the major challenges. Previous work has not examined the integration of science content based on the school curriculum in the gameplay mechanics of a computer role playing game (CRPG) and in a storyline which ends with a dramatic conflict. In this study, an educational CRPG was designed for learning concepts in the physical sciences according to the curriculum of the correspondent book. We integrated the content into the gameplay which included several tasks to be performed and puzzles to be solved, advancing players to successfully complete the game, according to the game’s storyline. Finally, we performed a usability test for ease-of-use and enjoyment issues. Most testers considered the educational CRPG entertaining. Computer role playing game’s gameplay mechanics provide affordances for the integration of science and technology courses in a playful learning environment.

1. Introduction

Video games are a popular medium which have appeared in several forms and configurations following the development of digital technology. While entertaining video games feature only the “game” dimension, serious games can be defined as “a mental contest, played with a computer in accordance with specific rules, that uses entertainment to further government or corporate training, education, health, public policy, and strategic communication objectives” [1]. Serious games provide an alternative way to practice real-world situations with safety and less cost and time [2]. Several efforts have been conducted in the direction of implementing serious games in the field of education. The first computer games for learning belong to the category of edutainment, using conventional learning theories and providing a simple gameplay with reference to certain curriculum [3]. However, educational serious games need to provide skills, such as problem solving, hypothesis testing or strategizing instead of memorization and repetition [4]. From this viewpoint, they can have perceptual, cognitive, behavioral, affective, and motivational influence in several educational fields [5], and especially in the field of science education [6]. However, the integration of the learning content into the gameplay mechanics needs very careful consideration [7]. Digital games which belong to a specific game genre, such as computer role playing games (CRPGs), include elements that are repeated [8]. Although there are educational serious games which use some CRPG’s gameplay mechanics, we did not find any serious games which integrate content from the physical sciences, and especially, chemistry into these specific gameplay mechanics. This viewpoint could provide useful information for game designers and perhaps stimulate more research on this topic.

Our intention was to use CRPGs’ mechanics in order to integrate content from the physical sciences into an educational serious game—the Adventure of Sciences (AoS). Players would control a virtual character (VC) and interact with the game world in order to accomplish a series of challenges. The successful completion of all the challenges will be necessary, as they benefit players with the knowledge and skills in the broader context of the game’s storyline. Gradually, those challenges will guide the players to the end of the game, providing a sense of character progression. At the same time, players’ performances will be recorded through a scoring system, which may be used as a tool for self-assessment and comparison [9] but without affecting the successful end of the game or discouraging players from continuing to play [10]. Feedback will be given in the form of text messages from several non-playable characters (NPCs) in order to instantly reward players or to assist them in their effort [9]. Moreover, we will conduct a usability test regarding ease-of-use and enjoyment issues in order to underline potential failures in the game’s design and to apply improvements, in a repetitive process. We will design AoS to be used in formal learning settings, as a complementary educational tool for students and teachers. The detailed representation of the content integration in the CRPG’s game mechanics could be used in order to clearly identify the process of designing and developing educational CRPGs. The educational video game and a YouTube video of the gameplay will be available online for use (See Supplementary Materials).

The paper is organized into the following sections. Section 2 outlines the related work and Section 3 describes the method. Then, Section 4 describes the results, and finally, Section 5 concludes the study.

2. Related Work

2.1. Design Principles for Educational Serious Games

Designers of good video games employ several methods for encouraging people to learn long, complex, and difficult games in a joyful way [11]. The potential of designing motivational game-based learning environments has been studied by several researchers and various game features could be employed in this direction.

A narrative context could be applied in order to situate the learning activity and to establish the rules of engagement [4,10], as educational content is often abstract and separated from everyday situations [12]. However, this narrative needs to be appealing for both boys and girls [12]. Moreover, the use of challenges, goals, objectives, and feedback/outcomes could boost players’ motivations and satisfaction [13]. Especially, employing a rewards system makes the game more enjoyable while reducing potential game-related boredom and anxiety. Nevertheless, the value of a rewards system is highly connected to comparison purposes or social interactions, helping with advancing in the game and releasing new content [9]. Finally, players experience the game world using VCs, which have a double duty as the self and as the other. As the self, they are connected to the players using interfaces, such as mice or keyboards, and as the other, they can be more efficient than the players themselves [14]. Giving players the opportunity to pick the character that will represent them in the game may inspire them, especially males players [15]. However, the critical challenge for educational serious games is that game designers might fail to apply pedagogical principles and educational practices to their video games, and at the same time, educators or instructional designers might design games which are neither fun nor engaging [16]. Four design principles have been suggested for the design process of math games in order to support both entertainment and learning: an engaging storyline; a constructive trial and error method to foster learning; collaboration; and the use of popular game mechanics to maintain the entertaining character of the game [10]. Studying the effects of specific game mechanics on the integration of educational content could provide useful information related to users’ experience and content learning. This could serve as useful information for game designers and educators.

2.2. Aspects of Design Related to the Mechanics of the Game and the Game Genre

In the digital world of mobile phones, computers, and various devices including toys with features such as memory, voice recognition, and interactive connectivity, the act of playing has been transformed [17]. The potential of educational serious games is based on the addictive nature of gameplay which attempts to facilitate learning in various educational fields, such as history, mathematics, or biology [18]. The most frequently reported games used for learning are simulations followed by puzzle games [5]. Nevertheless, strategy, sport, adventure, board, role playing games (RPGs) or action games have also been used for designing educational serious games. However, different game genres provide players with different experiences, a parameter which could be further examined. The game genre conveys directly what kind of user experience it delivers. Because of this, genres provide an appropriate perspective for us to understand and analyze different interactions in various types of games [8]. Therefore, the integration of educational content in the gameplay mechanics might be of great importance and needs further exploration. Simulations are open-ended worlds with many interacting variables. In these worlds, each participant takes a role and experiences the effects of his decisions [19]. Puzzle games present a problem that needs a solution by solving enigmas, navigating, manipulating objects etc. [20]. Action games involve challenges that demand from the player quickly reaction, hand-to-eye coordination, and spatial awareness of the game environment. Those challenges are usually time-critical, and they need quick decision making, according to the game world’s changes [21]. Action games can be played from the players’ view or using visual characters, fully visible to the player that connect him with the game world [22]. Finally, RPGs are games in which players create or take on a character whose capabilities can change, incorporating primarily exploration gameplay challenges and secondary logistical, conflict or tactical ones [23]. Players can freely navigate in the game world; therefore, each player’s experience of the game can vary significantly [21].

Especially in CRPGs, the player interacts with a fictional game world by controlling a character, as CRPGS are single player games. Therefore, the computer needs to facilitate the players’ actions within a fantasy environment by creating the game world for all the actions to take place, based usually on storytelling with rules. Moreover, there are clearly defined goals which must be reached by the player in order to end the game. Therefore, successful achievement of the final goal depends on the completion of several tasks which provide players with potential choices. However, these choices are strictly limited by the pre-programmed content [22,24]. In CRPGs, a character’s progression has the meaning of the player’s progression. Through the in-game actions, players earn points, gain levels, and thus, they can have access to new items, perks, and bonuses. Players can also take advantage of their progression to visit game areas that are no longer challenging for various reasons, e.g., to be engaged in risk-free character progression [23]. Success measurement is usually conducted through an experience-based point rewards system [9]. The different game genres, such as CRPGs, might offer different options that could be very carefully considered in order to design serious games to be used as educational tools.

2.3. Methodologies of Design for Educational Serious Games

The theorization of the cognitive processes which occur when a person plays a video game contributes to a better understanding and results in an effective educational serious game design and development process [4]. Several methodologies have been proposed in order to support the design of educational serious games which have the potential to motivate and enhance learning. The integration of common instructional systems designs (ISDs) in the game development process is a systematic process with the aim to optimize game-based learning [16]. From another viewpoint, game designers could apply a framework for the design of learning, storytelling, gameplay, user experience, and technology components of a serious game [25] or they could use an assessment tool for serious games—the serious game design assessment framework [26]. Such methodologies focus on specific aspects of the design and development process, such as the game and learning mechanics, and the interconnections between gameplay and learning [27], or concern scientific topics, such as software engineering [28]. An interesting idea is the use of graphical notations in the educational serious games’ development process as wells as the balance between the overall and the detailed views which are required to create a game, using various levels of abstraction [29]. From another viewpoint, an architecture which supports the design, execution, monitoring, and adaptation of collaborative learning processes for managing educational multiplayer video games, [30] or a model driven architecture approach (MDA) for the implementation of generic concepts within an existing serious game platform and e-learning environment [31] could be used depending on the different needs. Finally, the activity theory-based model of serious games (ATMSG) could also be applied on an educational serious game’s design process [32].

The ATMSG model [32] utilizes the conceptual framework of activity theory, describing the structure of educational serious games with an emphasis on the relationships between serious games components and the educational goals of the game. From this perspective, serious games and their users are complex and dynamic systems which can be described through at least three activities: gaming, learning, and instruction. In particular, the gaming and learning activities are performed by the player/learner (subject), using the serious game as a tool but from different motives, e.g., playing or fulfilling the course requirements. The instructional activity shares the same tool, which is the serious game, but it could refer to the instructor or the game designer whose motives are different. Finally, there is a distinction regarding the instruction activity which could be intrinsic or extrinsic, depending on who performs the activity, e.g., the game using messages and giving feedback or the instructor giving instructions before or in the end of the game. Moreover, activities are divided into sequences of actions mediated by tools with specific goals. The ATMSG model’s implications are the systematic and detailed representation of an educational serious game and the description of the various game elements, their connections to each other throughout the game, and their contribution to the achievement of the desired pedagogical goals.

The detailed representation of an educational serious game seems to have a significant influence in terms of designing, evaluating, and improving it. Most of the methodologies follow a generic approach for all types of serious games or focusing in certain game features. However, designers need better support in the educational serious games design and development process, and especially, the inexperienced. We decided to use the ATMSG model in the design process of an educational CRPG, applying some modifications in order to adjust the process to the CRPGs’ genre. The ATMSG model suggests the following steps in order to design an educational video game. In the first step, the playing, learning, and instructional activities must be described. Then designers must decide on the sequence of the game components, using a diagram in a rough timeline. In the next step, the game events must be analyzed by decomposing them into actions, tools, and goals in order to answer for each step: who is doing the actions, how, and why. The final step includes the implementation of the ATMSG model on a table, which describes the sequence of nodes and the gaming, the learning activities, and the intrinsic instruction activities if the teacher’s involvement is not described. Our intention was to follow the first step, describing the activities and their subjects. There was no need to perform and explain the second step separately, as CRPGs employ a gameplay which is based on a series of challenges which must be completed, following a specific sequence in order to successfully end the game. Therefore, the timeline can be simply described from the table’s structure. This cannot be influenced from the players decision to explore the game world, as that decision is not connected with the advancement of the game plot. Then, we could also skip the third step and directly prepare the implementation table, describing the sequence of the tasks and the answer to “who” in the column with the intrinsic instruction activities, the answer to “how” in the column with the gaming activities, and the answer to “why” in the column with the learning activities. The NPCs support through various mechanics must carefully be described in the intrinsic instruction activity too. Additionally, the character’s progression with skills and bonuses could be described in the gaming or the learning activities column depending on the nature of the advancement. We believe that these choices reduce complexity and time, giving a detailed description of the game, and resulting in an implementation of the ATMSG model for educational CRPGs.

2.4. Serious Games in the Context of the Physical Sciences

The major challenge of educational serious game design is to make learning and playing work together, or, at least, not one against the other [3]. Most of the educational serious games in the context of science education are simulations and puzzles [5]. The potential of incorporating games beyond simulations and puzzles could boost student interest and enhance learning. Game designers need to carefully consider which educational tasks, activities, and operations will be integrated into the gameplay and their relationship with the learning outcomes [5].

The literature provides information for titles designed for students of primary and secondary education, in the context of physical sciences, and especially chemistry. Although simulations provide great opportunities for science learning through interaction with several variables in open-ended worlds, we will focus on serious games which expect players to win, applying knowledge and skills [17]. In the same direction, we will not be engaged with games such as puzzles or games with no genre specification. On the other hand, multiplayer games such as Minecraft Education [33] or Legends of Al-khimia [34] provide multiple opportunities for collaborative learning of science content. Despite the great potential of such types of games, the technological settings might be unaffordable for the typical school environment [35]. Therefore, we decided to focus our study on single player games (see

Table 1. Examples of single player games in the physical sciences.

Game’s Name	Educational Goal	Game Genre 1	Storyline with Conflict	Source
Al-Kimia 2	Attitudes towards chemistry	CRPG + Action + Adventure	√	[36]
AKAMIA 3	Learning chemistry	Adventure + Action + Strategy		[37]
Alchemist 4	Assessment tool	Not defined		[26]

¹ Designers’ specifications of their game’s genre. ² Al-Kimia is a video game that aims to generate positive attitudes towards Chemistry. ³ AKAMIA is a chemistry mobile game-based tutorial. ⁴ Alchemist is a 3D video game.

).

Al-Kimia is a science game which applies features of CRPGs, action, and graphical adventure games [36]. In Al Kimia, the character moves inside a virtual world in order to explore and perform a series of chemical experiments, acquiring basic chemical knowledge. Moreover, he must overcome various enemies, acquiring the necessary skills to attack “evil Zosimo and his gang”. The game combines CRPG elements, such as the progression of a character or a storyline which ends in conflict with action games features. Although there is a rewards system based on the character’s progression and knowledge which must be acquired, action features, such as quick response to environment changes, give a mixed character to the game. Moreover, the knowledge which is provided does not follow the curriculum of a schoolbook. Nevertheless, Al-Kimia has the potential to capture players’ attention, providing a kind of science literacy.

AKAMIA is a video game for teaching chemistry based on short pop-quiz questions that need to be answered by the player [37]. According the designers, the game applies adventure, action, and strategy elements, providing a virtual world for the player to explore. Moreover, the player acquires knowledge, interacting with an NPC and answering questions related to various concepts from the correspondent book of chemistry. The player can fight a type of monster, after answering correctly a certain pop-up question. Although the game includes character progression, there is no storyline, while conflict concerns the right answer to a question. Moreover, the game’s plot is advanced through answering a series of pop-quiz questions. The game consists of an alternative way of learning chemistry in school, as the content is based on the curriculum of a schoolbook, but the player could be more motivated through a storyline and by having a more active role in the game.

Finally, Alchemist [26] provides a more interactive gameplay based on collecting information, solving problems, answering questions, and collecting virtual goods. In the end, the player must carry out some experiments, classify elements into acids and bases, and answer a set of questions related to the content. However, the gameplay is not based on a dramatic story with a conflict to be faced; the reason for playing is to learn and perform the final assessment test.

Such approaches incorporate elements of different game genres (Table 1) and provide alternative educational tools in the context of the physical sciences. However, we did not find any educational serious games which could be clearly classified as a CRPG designed to complementarily support science learning in school settings. The potential of implementing CRPGs may consist of an interesting design option, such as role playing activities, which provide powerful and creative tools in science classrooms for the understanding and teaching of abstract phenomena [38] and for creating meaningful connections with real-life applications [39]. Therefore, this study’s aim was to integrate educational content from the schoolbook in the gameplay mechanics and narrative of an educational CRPG to be used in formal school settings.

3. Materials and Methods

The intention of this study was to design a CRPG to be used as an educational tool for learning physical sciences concepts. The RPG Maker VX Ace engine was chosen for the design and development process due to its usability, graphical results, and low level programming skills required [40]. In the beginning of the project, decisions on the high-level characteristics of the game design were made.

Table 2. Game’s activities.

Activity	Subject	Description
Gaming	Students, 11 years old	The player is assigned the task to find and repair the sword which could defeat the evil dragon and bring peace again to the kingdom and its citizens. For achieving the final goal, the player moves in the game world performing tasks, solving puzzles, and acquiring knowledge and experience to prove his skills.
Learning	Students, 11 years old	The player needs to understand and practice the concepts of mixtures; homogeneous/heterogeneous mixtures; temperature; heat; the use of a thermometer; evaporation and boiling; foundries; chemical composition of water; molecules types, such as solids, liquids, and gas; melting points of various metals; salt pans; and the metal alloy making process within the foundry.
Intrinsic instruction	Game mechanics which guide players	The digital CRPG assists players through various mechanics, such as rewards and tips to achieve a better understanding of certain science concepts, according to the curriculum of the corresponding book.
Extrinsic instruction	Science teachers	The game will be used as a supplement by science teachers to assist in the learning process of science concepts.

describes the gaming, learning, and instructional activities which take place in the educational serious game and who will be performing them [32]. In particular, the gaming activity includes the challenges which are performed by the player in order to complete the game. The player gains knowledge and skills through their effort to repair the “legendary” sword and use it as a weapon against the evil dragon. Those knowledge and skills are related to educational content from students’ textbook on physical sciences [41], which is addressed to students who are about 11 years old. In particular, we used concepts on physics and chemistry from chapters in Mixtures and Heat in order to design the learning activity. The instruction activity concerns the game mechanics which assist players during the game (i.e., intrinsic instruction activity) and the teachers who use the CRPG to teach science concepts (e.g., extrinsic instruction activity). Nevertheless, the CRPG could be used in informal learning settings too.

3.1. CRPGs Game Genre

Players of CRPGs take the role of a VC that can be subjectively defined as a second self within the social and interactive world of the game [42]. Since the game is a CRPG, one playable character, who is a human VC, is controlled by the player in the game world. Moreover, there are several NPCs who cannot be controlled by the player but can still provide useful information for the game world via interaction with the VC. The special role of NPCs is to guide the VC through the various tasks to the end of the game. This interaction between NPCs and the VC provides an interesting opportunity for a player’s instruction, without removing him from the gameplay. This could empower better integration of the VC in the game world and provide an opportunity for educational content implementation. However, some of them have a role in just decorating the game world and empowering the exploration element of CRPGs. In a CRPG, the VC interacts with items, people, and places that exist in the game world. In that manner, when an NPC assigns a VC with a task, the VC must move in the game world and appropriate actions in order to complete it. Then, the VC will go to the next step, depending on the next set of instructions he receives from an NPC. A game’s progression occurs when the VC is assigned with a task/puzzle by the game’s NPCs and successfully completes it (See Figure 1).

The advancement of the game’s plot is based on the players, who can explore the game world and repeat the challenges as many times as it is necessary, in a constructive trial and error method which fosters learning [10]. This interaction with the game world advances the character, providing him with knowledge and skills, according to the narrative. The player is supported by the NPCs with advices and immediate feedback, and there is also the option of reading the related content of the physical sciences textbook. Moreover, the rewards system includes a chronometer and a score. The chronometer counts the time needed by the player in order to complete the game. It is a basic measure related to the successful end of the game, as many tasks must be repeated by the player until successful completion. Similarly, a player who has only given the right answers will traverse more quickly through most tasks. A player who make mistakes, or picks the right answers out of luck, will also have to repeat the test, thus burdening the time measure. The scoring system starts as equal to 0. However, it can be increased or decreased depending on the player’s actions and choices. The final score is an indicator of the player’s performance during the game. Thus, when the player’s decision is right, meeting one condition, the score increases. On the contrary, when the choice is wrong, the score is decreased. However, time and score have a secondary role and players can use them in terms of comparison and social interactions, if they want to [9].

3.2. Educational Content Integration

We used the ATMSG model [32] in order to design an educational CRPG and adjusted it to the CRPG’s genre. As mentioned, AoS employs a gameplay which is based on a series of tasks which must be completed and puzzles which must be solved by the player who acts from the position of the VC, interacting with the virtual environment and acquiring knowledge and skills in order to successfully end the game. The educational content is integrated within these tasks and puzzles according to the correspondent textbook [41]. Therefore, the sequence of these tasks and puzzles is serial and can be represented in the structure of a table. For each task or puzzle, we recorded the answer to who (Intrinsic Instruction column), the answer to how (Gaming column), and the answer to why (Learning column). The NPCs’ support through various mechanics is described in the Intrinsic Instruction column, while the player’s progression is explained in the Gaming or Learning column, depending on their nature. These choices reduced the complexity of the ATMSG model, adjusted it to the needs of the educational CRPG, and provided a detailed representation of the game flow (see

Table 3. Description of the CRPG’s implementation.

Sequence Node	Gaming	Learning	Intrinsic Instruction
Storytelling	Purpose of the game	-	-
1st task	VC explores the lab in order to find 3 mixtures	Mixtures	NPC introduces curriculum, assigns the task, and guides with feedback
2nd task	VC prepares his own mixture, looking for the appropriate materials in the lab	Mixtures	NPC introduces curriculum, assigns the task and guides, giving feedback
1st puzzle	VC answers a multiple-choice test related to the mixture he prepared	Homogeneous/heterogeneous mixtures	NPC provides a test assignment and feedback
3rd task	VC explores new locations to bring 3 mixtures to the lab	Mixtures	NPCs introduce curriculum, assign tasks, and guide the player with feedback and tips
4th task	VC tries to separate salted water, mud, and metal alloy mixtures in the lab, for the metal alloy separation, he must visit the blacksmith’s shop	Thermometer/evaporation/boiling/heat	VC is guided by the NPC and the game itself through feedback in order to complete the tasks and learn the content
2nd puzzle	VC answers a multiple-choice test	Evaporation/boiling	NPC provides a test assignment and guides the player with a tip to continue the plot
5th task	VC must find sand and fire-wood for the blacksmith, then he must wait for a while	Temperature/heat/global solvent	NPCs introduce curriculum and assign the player with two tasks
3rd puzzle	Meanwhile, another multiple-choice test is provided to the VC	Global solvent/sediment/solution	NPC presents new curriculum and provides a test assignment, feedback is also provided
4th puzzle	The VC is presented with an image that depicts the state of a material’s molecules. Then, he must choose between items that correspond to that image	Molecules types of solid, liquid, and gas, foundries, thermal energy/temperature	NPC provides curriculum and test assignment, they additionally give a tip and feedback
4th puzzle	The VC returns to the blacksmith’s shop and he is presented with more curriculum, then, he must choose the right fire for melting different metals	Melting points of various metals	NPCs provide curriculum, test assignment, feedback, and tips
6th task	The VC visits the frozen mountains to find and bring fire-wood and snow		NPC assigns the VC with the task to visit frozen mountain and bring fire-wood and snow
5th puzzle	VC visits salt pans, watches the process of salt making, and answers a multiple-choice test	Separation of salt and water through evaporation	NPC introduces curriculum, assigns test, and provides feedback
7th task	VC is re-forging the legendary sword, using the equipment and fire in the blacksmith’s shop	Metal alloy making process within the foundry	NPC guides the player to make the sword
8th task	VC fights with the dragon	-	-
Storytelling	End of the game	-	-

).

Therefore, we connected CRPG components with the gaming, learning, and intrinsic instruction activities of the educational CRPG, according to the ATMSG model. Each task and puzzle which is described in Table 3 must be successfully completed by the VC through his communication with the game environment and the various NPCs in order to advance the plot. Those tasks and puzzles support the learning parameter and they are tied to each other, leading eventually to the end of the game. Moreover, there is a repetitive testing of the acquired knowledge during the gameplay. We based some of the tasks and puzzles on the curriculum in the schoolbook and used some others due to their contribution in practicing the content in a way that could not be done in class. An example is the knowledge gained through class in the metals’ forging (Table 3). Finally, a book which is in the game world is an element which provides the players with more information on the educational content according to the student book structure for the players who need it. This option might be supportive for both student understanding and game plot advancement. When the VC completes all the tasks, the game ends. Finally, the prominent feature of the game’s design is the integration of the learning content into the narrative. As the learning content provides solutions to scientific and everyday problems in the real world, as it does in the game world, it also provides entertainment in the game. The player must achieve an understanding of the learning content, in order to advance the game.

3.3. Supplementary Design Decisions

3.3.1. Levels and Environment Design

After a small storytelling at the beginning of the game, the VC appears on a map and he is prompted to go north to the lab. Inside the lab, an NPC, the wise man, assigns him with the first task. Gradually, the VC is guided by the wise man and other NPCs through several tasks with an increasing degree of difficulty, according to the curriculum in the schoolbook (Table 3), conducted in order to explore the game world and perform the challenges. All the assigned tasks must be performed, and, in the same way, all the puzzles must be solved. However, the player is supported in his effort through rewards and tips and encouraged to repeat them if needed in order to acquire the necessary knowledge and skills to complete the game. The game world is composed of several locations, such as the lab, the village, and the swamp where the challenges are performed and for exploration. The game world contains objects and decorations which reflects the various contents. For example, the village contains small houses, farmers, animals, trees, etc.

3.3.2. User Interface and Game Controls

The user sees the game world from a bird’s view. Additionally, the score is displayed on the top of the screen and the duration time of the game can be seen by pressing the <Shift> key. As the game begins, the VC can be moved freely on the map, exploring the game world. This movement occurs through the arrow keys on the keyboard or the gamepad. The interaction with the game objects can be done by moving the VC over an interactive game object and pressing the <enter> or the <space bar> keys.

3.3.3. Aesthetics

The graphics and soundtracks which play as a musical background in the game are also very important. Some of them can be imported while others can be selected from the game engine’s library. For example, the images in the game’s introduction and ending can be imported in order to be more impressive. Also, the images for the mixtures and substances in the lab can be imported in order to provide students the chance to see the element that they are picking up rather than having only a written reference for it in a dialogue box. This could be beneficial for student learning in an early stage of the game. Therefore, images for lentils, beans, coffee, orange juice, and others can be imported, as well as photos depicting the processing of steel to make a tool at the forge. Nevertheless, AoS mostly uses the application’s built-in graphics. In general, our intent was to have a game world that would offer freedom of movement for the player and would not scare new players by proposing too much content.

4. Results

4.1. Game Development

4.1.1. Game Engine

The aim of this research was to design and develop a digital SP-RPG in order to be an educational tool that assists players/students in achieving a better understanding of science concepts, especially on mixtures and heat. For this purpose, we chose to develop AoS using a commercial RPG game development engine, the RPG Maker VX Ace, due to its usability, graphical results, and low-level programming skills required. The software belongs to the RPG Maker series, a series of commercial programs for the creation of RPGs, which has been extensively use by indie game developers and professionals. It was considered that employing this commercial tool would help in distributing the time provided for its completion towards the tasks of integrating educational content while at the same time also making an aesthetically pleasing game. In addition, the software provided an easy and accessible working environment [40].

4.1.2. Development Process

The game world of AoS consists of Locations, Characters, and Events. As a start, several locations were designed for the VC to move, explore, learn, and practice the various concepts. In addition, NPCs were also designed in order to guide the VC and advance the game-plot according to the relevant educational concepts. The lab is the more “content-heavy” area of the game, with a variety of mixtures and substances for the player to choose from. It simulates some of the experiments that can be done in a real laboratory. The NPC who acts in the lab and has a central role in the game process is “the wise man” (Figure 2).

Another location that can be visited by the player is the village. The village provides a story location and a meeting place. The VC needs to find the blacksmith’s shop, which is in the village, through his interaction with various objects. The NPC who acts in the village is “the girl” (Figure 3).

The girl instructs the player and assigns him with tasks and puzzles. However, the girl appears in another location too—the frozen mountains (Figure 4). The educational content which is transmitted through the girl NPC is related to the concepts of salt pits, molecular nature of gases, liquids, and solids. Another NPC, the wood cutter, is in the frozen mountains and helps the player learn the concepts of water and temperature.

Finally, the blacksmith’s workshop provides the setting for a few of the important events for the game’s progress. It is the location where the player is taught the concepts of mixtures, metals, and heat (Figure 5). There are also two more locations, the swamp and the beach, which were designed for the purpose of finding a mixture from the VC during his travel to a seemingly unrelated place.

Moreover, a storyline tightly connected with the game’s activities and goals was implemented in order to make the game more motivating. The VC is moving within the game world in order to acquire the appropriate knowledge to practice his skills and to prove his abilities. These will result in the re-construction of a sword which will be used by the VC in order to defeat the evil dragon and save the kingdom (Figure 6).

Another important step is to define the events and switches. Once a switch is turned on, an event begins. Events start when some conditions are met. Those conditions are described in the event’s page in the form of switches and “self-switches”, which are internal conditions for the event to work. In any event, switches can be incorporated to set the requirements not only for whether an event will run or not, but also for its outcome. Almost everything in the game is managed through events, triggers, and switches. Any interaction the VC has with the world is attributed to events. When the VC walks to a location in the world map, an event that transports him to a location is triggered. Likewise, when the player attempts to initiate conversation with an NPC, after the player presses the “action” button, an event is triggered. Similarly, if certain conditions have been met, the event may be different than the last time it was triggered. More specifically, if a certain character asks the VC to collect a certain number of items, a switch is triggered. When the VC returns to the NPC, if the items mentioned have been collected, the condition of the event has been met. Therefore, the switch inside the event triggers again, and the dialogue and the game plot move forward. The events which were created within the AoS can be transportation from one place of the map to another, dialogs, picking up items, interacting with objects, etc.

4.2. Usability Test

Usability is an important aspect of game quality, but designing usable games is difficult. Video games are different from traditional software systems, as their purpose is not to support a user to perform some particular tasks, e.g., processing texts, buying from a website, managing appointments. Games have the purpose of entertaining and, for some of them, educating people in various contexts as well [43]. Therefore, we decided to examine AoS in terms of ease of use and entertainment before performing an evaluation test in terms of learning. This decision may assist in eliminating potential design failures, ensuring that students find the gameplay entertaining, and tracking the instructions which are confusing

4.2.1. Usability Test Setting

In a first effort to evaluate AoS, we decided to assess the game in terms of usability and fun. We asked science teachers from nearby schools to participate in the usability test, in order to have a first overview of the game. One science teacher kindly accepted to participate in the process. We decided to ask students to play separately in order to have enough time to observe them. Although nine students from a total of 12 wanted to participate, we finally performed the usability test with six students due to the time limitations. Nevertheless, we preferred a better image of the process instead of using a larger sample in order to see and discuss details related to the usability of the game and its entertaining character. Additionally, students were already taught the educational content from previous lessons at school. Their teacher was also experienced in similar lessons.

The usability test was performed at a middle school in north-western Greece. The school is located in an urban area and may be considered typical in terms of the number of students, their reason for attending, and the school’s infrastructure. The usability test was conducted from one of the researchers with each student separately. First, the researcher was asking the participants to play the game for 15 minutes and helping them during the play. Informal notes were kept during this process. Then, a questionnaire was given to the players in order to rate the ease-of-use and the enjoyment scores using the Five Degrees of Happiness Smiley Face Likerts (SFL) scale. After, the researcher asked questions in an informal semi-structured interview. The same process was followed for the teacher.

4.2.2. Sampling

This first phase of the usability test was based on a group of six students who were 13 years old and one teacher with experience in the relevant content. Five boys and one girl decided to play the game. Children between 11 and 14 years old can be included in the usability testing, as they are comfortable with computers [44]. Moreover, these students were already familiar with the educational content from previous lessons at school. We wanted students familiar with the content, so that they could focus primarily on the usability issues and fun. Participants were asked to participate voluntarily.

4.2.3. Measures and Data Analysis

As mentioned above, six students and one teacher with experience in teaching the particular educational content were involved in the usability test process. Student answers were rated regarding ease of ese (EU) and enjoyment (ENJ) with the Five Degrees of Happiness SFL scale, which is an effective method for children, encouraging the use of all the scale points and providing an effective method for children to deliver judgments [45]. The questionnaire which was used for the game’s evaluation can be found in Appendix A. Additionally, semi-structured interviews with the students and the teacher were performed in order to provide a more complementary picture of the usability test results. Conversations were conducted with all participants. Researchers encouraged them to talk about their experience of using AoS. In addition, information about student opinions and attitudes towards entertaining video games were collected. Finally, the teacher was asked to suggest potential improvements and to discuss the potential of using this game in learning settings. Informal handwritten notes of the answers were made by the researcher during these conversations. In the end, an inductive content analysis of these answers was performed: in the first phase, all the interesting phrases within the informal notes were underlined, while in the second phase, the study’s results were coded in order to develop the study’s coding schema. The study explored coding categories, such as the player’s support, entertaining suggestions, and the potential of using the game for learning from the students’ interviews, while the content integration and intention to use the game were investigated through the teacher’s interview (

Table 4. Inductive content analysis.

Coding Categories	Examples
Player’s support (students)	“it does not explain the buttons”, “I need more instructions”, “instructions are tiresome”, “I did not know what to do next”
Use for learning (students)	“good idea”, “good way of learning”, “I don’t think I can learn this way”
Entertainment suggestions (students)	“more action”, “more random events”
Game content integration (teacher)	“better processes representation”, “not answers yes or no”

).

4.2.4. Findings

First, we quantitatively examined student answers in the Five Degrees of Happiness SFL scale. In particular, we examined the EU score (M = 2.8, SD = 0.4) and the ENJ score (M = 3.6, SD = 0.49); EU was rated at 56% and ENJ was rated at 72% (Figure 7).

Then, we qualitatively examined student interviews and we triangulated the results with the observations during the evaluation process, obtaining some interesting outcomes. Most students consider AoS entertaining. Moreover, students who often play video games for entertainment made some suggestions. They wanted more action during the gameplay and various random events that needed instant reaction. When researchers asked them what types of video games they prefer to play, they mentioned first shooter games, action games, and multiplayer RPGs.

On the contrary, students complained about usability issues. The instructions which were provided and the player’s support during the gameplay were the main reasons. Most of the participants needed more guidance during the game, as many times they “could not figure out what to do next”. In the same direction, a girl, who was not a gamer, could not even start the game and was only walking around the game world. Another boy said, ”I do not like reading all the instructions”, and he was trying to explore the next steps by looking for clues around the game world. Another student argued that there were too many instructions in the game, making the process tiresome. The researcher’s involvement helped students to quickly overcome such problems and supported the less experienced students in playing and exploring the game world. Most students were positive with the implementation of educational CRPGs for learning, except one who was doubtful about their effectiveness compared to traditional settings.

Finally, the teacher made some interesting observations related to the learning parameters of the game; she suggested more descriptive chemical processes should be displayed for the students and a better balance between the game’s duration and the quantity of the content should be observed. Overall, she was positive in using the CRPG in her class as a complementary and playful alternative educational tool, if the time restrictions allowed it.

5. Discussion and Conclusions

The aim of this study was to design an educational serious game to be used for learning concepts in the physical sciences in formal learning settings. However, the integration of learning with playing activities is the most critical challenge in the design of playful learning environments. We decided to design AoS using popular gameplay mechanics in order to maintain its entertaining dimensions [10]. There are various genres offering different experiences to players depending on their gameplay mechanics, such as simulations, action games, strategy games, etc. We decided to use CRPGs’ gameplay mechanics due to their features, e.g., “first-hand experience and absence of risks which enable players to be engaged in unlimited experimentation in situations that encourage significant learning” [39]. Previous serious games in the field of physical sciences, and especially chemistry, have not employed basic CRPG gameplay mechanics. We based AoS on a series of linked challenges which can be considered a narrative [12]. The player needs to complete those challenges which include tasks to be performed and puzzles to be answered through their interaction with the game world and other characters. This storyline gives the players the opportunity to acquire the necessary knowledge and skills, proving their abilities, and completing the game. In that sense, the character progresses according to the CRPG’s gameplay mechanics. Additionally, we integrated into this narrative a dramatic story which concerns a great evil which must be defeated by the protagonist through a series of obstacles and challenges, using a type of weapon [40]. Such game elements could be used from CRPGs in order to motivate players. Therefore, AoS guides the protagonist to acquire knowledge and skills in order to repair the legendary sword and defeat the evil dragon. Another important gameplay mechanic concerns the use of feedback and tips which guide and help the player in achieving the final goal while the scoring system and the chronometer can also be used for purposes of comparison and social interaction. Nevertheless, the player can repeat the assigned challenges without any influence on the successful end of the game, in a constructive trial and error method [10]. Finally, AoS integrates the curriculum of a correspondent textbook for physical sciences for students who are 11 years old. The player needs to explore the game world, to perform tasks, and to answer puzzles in order to repair the legendary sword, performing the metal alloy making process. Then, he can use it as a weapon against the evil dragon. Tips and instructions assist him in his effort. Therefore, we consider that the AoS is an educational CRPG which includes basic CRPGs mechanics and features [40], promoting meaningful play, play which connects what a player does with how the game responds to it, applying actions and outcomes which are discernable and integrated in the game context [46]. In summary, we designed an educational RPG which employs narrative, a dramatic story with conflict, the player’s progression in terms of learning, and a rewards system which involves tips, rewards, score, and chronometer. We integrated formal curriculum into the gameplay mechanics. Similar games in the context of physical sciences, and especially chemistry, such as Al Kimia, Alchemist, and AKAMIA, have employed some of those elements. A narrative with a dramatic story and conflict is used by Al Kimia. A rewards system is employed by all the games in various ways. However, Al Kimia does not use formal curriculum and AKAMIA employs questions that must be answered but it does not provide active learning. Finally, all games combine elements from various genres according to their designers.

Applying a common language in order to design and represent a serious game can be useful in terms of understanding, evaluating, and improving the game. Although Al-Kimia’s design was based on a hybrid framework, most of the games within a science context do not systematically represent the integration of the content in the gaming activities. We represented the AoS design based on the ATMSG model, which describes how the game elements are connected to each other throughout the game, and how these elements contribute to the achievement of the desired pedagogical goals [32]. However, we made some modifications in order to reduce its complexity and adjust it to the needs of the CRPGs’ game genre. In the beginning of the design process, we described the gaming, instruction, and learning activities, their purpose, and the subjects who are going to use them. Then, we represented the overall design by describing what is being done at each point of the game, using which tools, and for which purpose.

Usability is an important aspect of a video game’s design. Moreover, video games which are not entertaining fail to engage players [43]. Therefore, we evaluated AoS in terms of usability and enjoyment before examining the learning parameters. Regarding the entertainment parameter, all students enjoyed playing the educational CRPG during the usability test process. However, some of them made some suggestions, e.g., more action and more random events to be handled during the gameplay. Those suggestions could be explained by students having prior gaming experience and skills, as they were gamers who like to play action and first-person shooter games. There is evidence that there are differences in the game-behavior of first shooter games compared to non-gamers and RPG gamers in terms of time needed to solve puzzles in 3D games [47]. Therefore, this prior experience and skills might be responsible for such suggestions. Nevertheless, we did not notice any complaints related to the enjoyment parameter during the usability test process (Table 1). On the other hand, most students who played AoS mentioned usability issues, as they needed better guidance during the gameplay. We examined those usability issues very carefully, as the overall usability of a game combined with in-game tutorials and assistance seems to have a critical role in order to allow new players to feel more comfortable and confident in exploring/using the game world [48]. It is possible that our choice to use the text communication did not support players in order to advance the gameplay quickly. Educational serious games could employ visualization and problem-solving based challenges [49] in order to be more effective. In future steps, we are planning to reduce text communication and use, instead, multimodal texts, images, and tutorials for supporting students in playing and learning activities. Finally, the role of the teacher during the activity can be of great importance, guiding students in playing and learning. Computer games may be successful educational tools, but they have clear limitations, and therefore, qualified teachers should guide such didactic interventions in order to facilitate learning [50]. Nevertheless, most students were positive in using such interactive educational tools in class. Educational serious games provide learners with experiences in a virtual world, making learning engaging and pleasant [51]. AKAMIA was also evaluated in terms of usability; users seemed to be satisfied with the game’s interface as it was not confusing or misleading. From another viewpoint, Al Kimia was evaluated regarding the relationship between chemistry and people’s health, emotions, and everyday life, as the game’s aim is to promote positive attitudes towards chemistry. Finally, the Alchemist was evaluated as an assessment tool by experts; the game seems to be valid to assess the problem-solving competence of the users. In the same direction, we are planning to evaluate AoS as a learning tool in a future phase of the design and development process.

The design of AoS concerns content from a textbook on physical sciences for students who are 11 years old. Therefore, the generalizability of the design process must be carefully considered. More CRPGs gameplay mechanics could be applied, such as a stronger character progression element through item collection and bonuses and avatar choice options due to the strong connection of this feature with the player. The usability test was performed with a very small sample and the players had the limited time of fifteen minutes to test the game in terms of usability and entertainment. The results of the usability test must be confirmed through more tests. Nevertheless, our intention was to continue our effort to improve the game and the design process of similar games in the context of the physical sciences. Moreover, we will perform more tests in a repetitive process with more players for longer periods and for many parameters, including learning. Finally, useful outcomes can be drawn from evaluating AoS with more teachers as players in order to examine their intention to apply such methods in their classes, their suggested improvements, and their expectations of the usage of such educational tools.

The implication of this study is the design of a playful educational CRPG to be used by students and teachers for learning of the physical science concepts of mixtures and heat in a school setting. This curriculum is included in textbooks for physical science students in elementary school who are 11 years old. The design process of the educational CRPG was described through the ATMSG model after adjusting it to the needs of CRPGs for reducing complexity and time and making the representation of AoS more detailed.