“Let Us Save Venice”—An Educational Online Maze Game for Climate Resilience

Abstract

Climate resilience competencies improve people’s capacity to recognize and adopt strategies for mitigating negative climate effects. Especially concerning the built cultural heritage protection in the coastal areas, both professionals and citizens have to be prepared for water-related extreme events, such as floods, sea-level rise, and altered precipitation. Considering these challenges, the authors explore the efficiency of serious video games and describe the process of design and validation of the pilot educational online maze game “Let us save Venice”. More specifically, the main research question motivating the study is to identify what the experience factors and the mini-games are that contribute mostly to raising awareness to climate resilience and built heritage sustainability. The question implies two main research objectives: first, to explore what role user-centered game design plays in creating immersive and enjoyable educational video maze games, and second, to investigate which factors influence user experience and game playability in the design phase, and how they contribute to learnability and raising awareness. The background section of the article focuses on a preliminary study of the educational potential of serious games and introduces the EU-funded project e-Creha. Then, it explores the process of game design, development, and validation, focusing on metrics such as game learnability and game experience. The results outline the main findings that immersion, positive affect, and competence appeared to be the main experience factors contributing to raising awareness to climate resilience and built heritage sustainability. Lastly, the discussion section provides further directions for game improvement and future work.

1. Introduction

The intensity and frequency of extreme weather events are among the key visible outcomes of accelerating climate change. Water-related events such as floods, rising sea-levels, and altered precipitation already put at risk a considerable part of the coastal areas [1]. The rising waters threaten to invade not only public infrastructure and roads but to damage unique architectural heritage and cultural monuments. Among the numerous endangered places is the iconic city of Venice, which suffers from increasingly frequent and higher floods (Aqua Alta) [2]. Climate change and extreme weather further threaten the lagoon and more specifically the built heritage of Venice. In the same place, several ambitious climate resilience and mitigation projects are undertaken in the field. For example, the MOSE project [3] promotes proactive strategies for coping with high floods, generated by extreme weather conditions. These initiatives have to be properly studied and popularized amongst the larger community of professionals: architects, urban designers, building engineers, city planners, technical experts, and others, who need to better understand them and prepare the next climate resilience strategies [4].

The topics of climate change and climate resilience are still new in vocational education and life-long learning. At the same time, it is observed that serious games or games with a purpose other than pure entertainment have considerable impact especially in the field of climate change education [5]. By making a balance between learning and fun, serious games for learning are recognized as a powerful and persuasive tool, encouraging participation, ideation, and discussions among stakeholders. Covering a wide range of forms and mediums, serious games can play a significant role in community building and engagement [6], in the promotion of participatory design [7], in raising risk awareness, and in stimulating decision making and behavior change [8]. Games also can build capability and capacity to train in safe innovation spaces and interactively engage with alternate climate futures [6].

Some authors such as in [7] state that climate change games typically have three primary objectives: teaching knowledge and providing familiarity with the issues of climate change; making players aware of the challenges associated with global warming; and encouraging players to develop solutions. However, ref. [5] underline that the potential of serious games in climate change is not limited to education or raising awareness. Serious games can be increasingly efficient in the life-long learning context and for reaching a larger target audience. They find that well-designed games can stimulate immersion, create long-lasting memories, and encourage creativity, critical thinking, collaboration, communication, and problem-solving skills.

Considering that educational video games still represent a small part of the games dedicated to climate change, the present research aims to explore the design and implementation of the serious video maze game “Let us save Venice”. The game was created within the European project e-Creha. The e-Creha project [9] is an Erasmus+ project dedicated to designing and implementing educational solutions for raising awareness and life-long learning for the vulnerability of built heritage to climate change and the development of climate-adaptive strategies for a resilient society. The video game was generated by means of the APOGEE game platform [10] and is a 3D educational maze with five halls containing mini-games of different types representing didactic tasks.

The research question that motivated this study is as follows. What are the experience factors and the mini-games that mainly contribute to raising awareness of climate resilience and built heritage sustainability? This question implies two main research objectives:

• To demonstrate how user-centered game design can be applied for the creation of an educational maze game dedicated to climate resilience; and
• To evaluate a developed serious maze game with regard to user experience and game learnability.

The structure of the article is as follows. The first part gives an overview of the main theoretical findings for serious games types, functions, and challenges, focusing on educational video maze games. Then, the background settings of the study are introduced, revealing the e-Creha project aims and objectives and preliminary state of the art analysis in the digital games for climate change and/or built heritage. After that, the theoretical concepts of the user-centered design in serious maze video games for learning, which served as the basis for game design, are described. The research methodology section discusses the steps for the design, development, and field trial of the game “Let us save Venice”. Next, the results section explains the process of game validation, exploring the outcomes, data, and visualizations of the results. Lastly, the conclusion section provides a summary of the next steps and sets the path for future work.

2. Background and Related Works

2.1. Serious Games

The term serious game has been used frequently in recent times, and different authors ascribe different meanings to it. One of the first to introduce the concept in the 70s of the last century was Clark Abt [11]. His definition of the term serious game refers to all types of games, including traditional outdoor games, board games, role-playing games, and computer and video games. According to him, serious games have an explicit and carefully considered educational purpose and are also not intended to be played primarily for entertainment. Next, he asserts that this does not mean that they should not be fun and enjoyable [11]. Subsequently, ref. [12] redefines the concept of the serious game by updating the definition. The idea is to connect a serious purpose to knowledge and technologies from the video game industry. One of the more general definitions that have been proposed is “games whose first purpose is not just fun” [13]. Among the more detailed definitions, the following can be noted: “A computer-challenging brain involving specific rules and based on entertainment to achieve goals related to institutional or vocational training, education, health, domestic policy or communication” [14]. A more modern definition is given in [15]: “A Serious Game is an artifact, digital or otherwise, for which the original intention is to combine with consistency, both serious aspects such as non-exhaustive and non-exclusive, teaching, learning, communication, or the information, with playful elements from the game”. Usually, this is achieved by embedding the practical knowledge and functions in the key game components—plot, graphics, and audio. Thus, the resulting game is focused not only on pure entertainment.

Serious educational games have already been shown to be valuable in the context of climate change and sustainability due to their potential for increasing the personal and emotional engagement of the game’s players [16,17]. Such games also can enhance the focus on these essential factors of complex issues such as climate change and sustainability through purposeful abstraction, reveal hidden internal relationships between them, and bring to systematic view and understanding the topic as a whole [18]. Hence, serious educational games allow the transformation of complex problems into tangible ones. Thus, they are an appropriate tool for raising awareness and improving communication regarding environmental problems and sustainable development within various audiences [19]. As such, well-designed serious educational games have the potential of not only mere entertainment; they offer the possibility to learn about dealing with climate challenges.

Usually, serious games are classified according to several criteria, such as market and/or purpose for which they are intended. Further, with the expansion of video games, a classification reflecting the type of interactions involved is used. The so-called genres are action, adventure, shooter, racing, etc. The introduced “gameplay/purpose/scope” (G/P/S) model proposes classification criteria suitable for any video game type, including serious and entertainment ones [19]. In the context of climate change, these classifications are not directly applicable. Authors such as [5] present a comprehensive review of serious and simulation games on climate change and propose a new typology for their classification. It applies some generic dimensions and characteristics suitable to any game and some specific ones (topic, system-level) appropriate for a particular topical area. In addition to the general dimensions that are used (authorship, stated purpose, target audience, player interaction, medium, etc.), simulation, role-play, game-play, abstraction, and others are also proposed.

Serious games can provide different settings and help develop various skills [20]. As substantial research proves that they are very efficient in teaching and learning, such games are widely used in many areas such as education, military, healthcare, etc. and in society as a whole. Serious games are used as an educational resource already in many schools [21] and universities. They can have multiple educational purposes, including knowledge delivery and testing [22]. Furthermore, they are efficient in the specific training of the wide audience on issues such as battling fake news and misinformation [23] and increasing cyber-security awareness [24]. During educational gameplay, players gain experience in different settings and situations, acquire contextual knowledge, apply decision-making strategies, and develop social skills. Moreover, there is the possibility for players to test the knowledge they acquired in the same game context, by making trials that otherwise would be impossible in the real world due to reasons of safety, time, or cost constraints.

2.2. The E-Creha Project and Background Research on Digital Games for Climate Change and Built Heritage

The project e-Creha (education in Climate Resilient European Heritage Architecture) [9] aims to develop a multi-disciplinary knowledge for building a culture of prevention and mitigation (culture of preparedness) regarding climate change. In this perspective, the project focuses on increasing the understanding of how built heritage and design can strengthen the societal awareness of climate change and to show how heritage and design can play an effective role in shaping vulnerable environments while preserving them.

The e-Creha project activities focus to create a joint platform of collaboration, co-learning, and cooperation among six partners, pursuing four strategic actions/objectives:

• Promote an international course on climate-resilient architectural heritage as an incubator of innovation and backbone for experimentation of new curriculum courses;
• Provide pedagogical training for new tutors through peer teaching method;
• Nurture a collaborative and interactive flexible learning environment for tutors and learners, and strengthen interactive dialogue between partner universities and the wide community;
• Structure an interactive knowledge, research, and experience exchange through an e-Creha e-learning course that will draw a wider readership on climate resilience.

During the first phase of the e-Creha project implementation [9], project partners researched the available educational resources, learning games, educational projects, and publications in the field of climate change, climate resilience, building cultural heritage, and monumental cultural heritage protection. During the study, the authors found only 37 video games covering topics of climate change or/and built heritage. This number is relatively small and clearly insufficient to reach a bigger audience and raise awareness on the subject. In this perspective, considering that most of the games are available only in English, their impact could be limited to attract more efforts in the field of climate change and climate resilience. The search process was conducted in the period February–March 2021 by using the most popular search engines (Google, Bing, and Baidu), together with Web of Science and Scopus. The investigation of the games is performed considering several aspects such as the game type, dimension (2D or 3D), game genre, and online resources. The study was performed using different combinations of the following keywords: {game(s), mini-game(s), puzzle(s)}, {climate, climate change, environment, environment protection}, {heritage, built heritage, cultural heritage, monuments}, {sustainability, resilience, vulnerability} in the languages of the partner countries (English, Spanish, Italian, and Bulgarian). The search queries also returned several descriptions of games matching keywords, though the games were unavailable because they were Flash-based or their host sites were expired or inaccessible.

For each digital game, detailed information for game characteristics was collected, e.g., about game objectives, a brief description of the core game loop, game mechanics, dynamics, aesthetics of the game, and others. Based on these data, several significant conclusions were drawn. As described in Table 1, the predominant genres appeared to be puzzles and action-adventure games. Nearly half of these games are dedicated to monumental cultural heritage (in fact, 15 games plus a collection of 28 games and mini-games about ancient Rome), while 21 video games are about climate change. After analyzing the specific knowledge domain, the authors found that most of the games address issues of resilience (73%), impact (49%), adaptation (35%), and vulnerability (14%). Most of the games are single-player (76%), and only nine games are multiplayer video games. Concerning graphics, 70% are 2D games, 27% of games support a 3D graphic interface, and one 2.5D. All games are available in English, and only 8 support other EU languages. These outcomes served us as the starting point for the design and development of the serious game “Let us save Venice”.

Table 1. The main characteristics and settings of 37 video games.

Topic		Domain		Genre		Graphics		Players
Climate change	56% (21)	Resilience	73% (27)	Puzzles	27% (10)	2D	70% (26)	Single-player	76% (28)
		Impact	49% (18)	Simulation	19% (7)	3D	27% (10)
Cultural heritage	44% (16)	Vulnerability	14% (5)	Action- adventure games	38% (14)	2.5D	3% (1)	Multi-player	24% (9)
		Adaptation	35% (13)

2.3. User-Centered Design in Educational Video Maze Games

The research on educational games can be viewed from three broad, overlapping points of view, where the emphases on games and players vary. The approaches to game creation can be game-centered, player-centered, and user-centered. The game-centered approach is focused mainly on constructing game design with an engaging environment and context, thus supporting the learning process [25,26,27,28,29]. The player-centered approach usually focuses on players, their preferences and perception, and the manners they participate and communicate, both within and beyond games. Thus, the emphasis is more on the social aspects of learning and learning content [27,28,29,30,31]. Much research explores players in massively multiplayer online games settings, where players collaborate and interact with each other based on playing rules, knowing the game’s social norms, and using game tools [32]. The user-centered approach considers games as part of a media framework that embraces various digital technologies and environments, where the perspectives of users and context of use are raised [33,34]. We argue that the user-centered design (UCD) approach is more related to the user experience, as well as to the main factors of the user experience in games, such as playability, usability, and learnability, as described in [35]. Furthermore, the UCD approach focuses on user behavior, user satisfaction, and general feedback from game users, in line with the application of certain quantitative and qualitative indicators on data concerning the processes of collection, analysis, and evaluation of data, about users and users’ behavior [35].

Serious games are a promising tool for seamless learning. However, teachers and domain experts without programming knowledge still lack easy to use instruments for their efficient creation [36]. This work presents an approach for structured support, from determining the game structure, components, and content design to final implementation. The developed serious educational game is based on task-based learning theory [37]. The players have to perform context-dependent cognitive tasks while exploring an information-rich maze virtual environment. According to us, such a structure is especially appropriate for cultural heritage and sustainability applications. The idea is to define a maze game—a set of linked halls enriched with embedded educational tasks, implemented as mini-games. Such an approach supports and simplifies the authoring process by using a tool for 3D maze modeling and implementation of various learning tasks. The methodology used for game creation is top-down, starting from maze structure based on an analysis of the game’s knowledge topic down to the key knowledge goals and associated tasks that are instances of sample, predefined types of mini-games, puzzles, and quizzes.

Based on the research and the presented results [35] and with regard to the application of user-centered design for the evaluation of user experience in educational video games, in this article, we step on [35] to further develop the user-centered design approach in the game development and evaluation life cycle of the APOGEE software platform for educational video games, illustrated in Figure 1. The bottom part of the diagram represents a complete user-centered design schema, i.e., it is a detailed view of the game development and evaluation life cycle of the platform. The APOGEE software platform includes multiple developed software instruments [10,38] for the creation and evaluation of video games for education, with specifically designed and developed functionalities, according to the application of the Taxonomy of Instruments for Management and Evaluation of the Design of Video Games for Education—TIMED-VGE taxonomy [39]. The platform includes assistive and analytics software instruments that support the management of the design and analysis of data [35]. The presented UCD approach and the instruments facilitate the processes of the APOGEE software platform and provide the opportunity for creating improved game design and user experience.

Figure 1. The user-centered design approach in the game development and evaluation life cycle.

The top part of the diagram depicts the main modules of the UCD, integrated into the game development and evaluation life cycle of the APOGEE platform. It comprises “a three-level process of collecting and analyzing data about the user and user gaming experiences” [35]. This approach has proven the positive effect on the design and creation of educational video games in the APOGEE platform. It provides the user-centered evaluation of the designed games and the evaluation of the learning and gaming experience of the users within the platform, measuring user experience main factors, such as playability, learnability, and usability. Hence, this successful UCD approach is applied to create, develop, and evaluate the climate resilience video game “Let us save Venice”.

The illustrated diagram in Figure 1 consists of two separate parts. At the top is a diagram of the general view of the UCD. It is applied within the platform for the development and life cycle of the game. In essence, this is a cyclical process in which the designer takes part by going through the various stages of design, creation, gameplay, analysis, evaluation, validation, and game design improvement of the educational games within the APOGEE platform. The designer can access the three main modules, which are interconnected. These are the modules for Creation of a game, Game Play, User Experience evaluation, and Gaming and Learning Analysis.

At the beginning of this section, we presented our concept of the UCD approach, which positions the user at the center of the process of designing and creating educational video games. Following the pre-defined educational goals and according to the characteristics of the learner, the design process follows the UCD approach to designing and creating an educational video game with the ability to personalize the educational content and adaptive user gameplay. The UCD approach has been successfully integrated into the APOGEE platform. It has proven to be successful in designing user-oriented personalized educational video games [35]. The designer applies the UCD approach in accordance with the analysis and evaluation of data about users, users’ behavior, users’ satisfaction, general feedback from game users, and so on to improve game design and create improved user-oriented educational video games.

The second part of the diagram presents a detailed view of the UCD approach implemented in the APOGEE platform. The three main modules at the top of the diagram are presented in detail with their constituent parts. The designer aims to start designing a training video game by accessing the “Game Creation” module. This module consists of four submodules, through which the process of creating the game goes. These are Primary Game Design, Game Formal Description and Generation, Game Tuning, and Game Build and Deploy. After creating and building the game, it is available to users to play and learn. All users of the system have the opportunity to play games that are designed and created on the platform.

There are four submodules in the second main module—Game Play and Survey—which are specially designed for the user. A particular preliminary training tutorial has been created for the players, which aims to acquaint the player/learner with the main goals of the game, its mechanics, and its description (submodule Pre-Game Tutorial). Then, they move on to the Game Play submodule where they play and learn in the educational content in the maze game. After the players/learners complete the game, they move on to the following two submodules. These are the Post-Game Questionnaire and Semi-structured Interview submodules.

These two submodules collect and analyze the data necessary for the third main module—Game Design Validation and Improvement. After completing the maze game, all players/learners go through the submodule Post Game Questionnaire to fill out an online survey, which is user-oriented. In addition, they take part in a semi-structured interview. All results and data obtained are analyzed in the submodule Analysis of Survey Results. Based on this analysis, an assessment is made of the developed game and its design to validate and improve (possibly) the game design (presented as submodules Game Design Evaluation and Game Design Improvement). If, after the user-oriented evaluation and validation of the design, there is a need to improve the design of initially created video game for training, the designer introduces the necessary adjustments and passes on the game with the improved design for a formal description and generation. In this way, the cyclical nature of the approach is preserved in the APOGEE platform, taking benefits from the advantages of the UCD.

2.4. Evaluation of Game Experience and Learnability

Evaluation of game experience and learnability is of great importance for the validation of serious games for learning [40]. Game experience has been defined as “an ensemble made up of the player’s sensations, thoughts, feelings, actions, and meaning-making in a gameplay setting” [41]. One of the great challenges in its evaluation through post-game self-report lies in methods for constructing and validating questionnaires such as the Game Engagement Questionnaire [42], Game Experience Questionnaire [43], and Gameful Experience Questionnaire [44]. Whereas some of them have only one dimension, e.g., the progression from immersion to presence, flow, and absorption [42], others have more dimensions such as the Gameful Experience Questionnaire [44], which includes six dimensions: enjoyment, absorption, creative thinking, activation, absence of negative affect, and dominance. The core module of the popular Game Experience Questionnaire, or GEQ [43], evaluates game experience as scores on seven components: (sensory and imaginative) immersion, flow, competence, positive and negative affect, tension, and challenge. For a robust measure, the authors included from three (for tension) up to six items (for immersion) per component. Thus, GEQ contains 33 items—and even 42 items in its original version [43]—addressing the seven components by applying a 5-level Likert scale, which may appear to be boring for many of the players while responding to the questions.

Besides game experience, validation of educational games should address the assessment of the game’s learnability as one of the most important attributes of usability together with effectiveness, efficiency, satisfaction, attractiveness, and others [45]. By definition, learnability refers to the characteristics of an interactive system that allow beginner users to get how to use it and then how to achieve the maximum level of performance [46]. Educational games can be considered an interactive system, so to have learnability, they should be easy to learn so that the user can quickly start playing. The length of time a novice user needs to become productive with the system, i.e., start playing, also influences learnability [47]. Naturally, easiness to learn how to play a new game will affect player attitudes towards the game. The specific measurable attributes of learnability are direct and indirect. In [46], the authors define five direct attributes of learnability—familiarity, consistency, predictability, synthesizability (user capacity to assess the effect of previous operations on the current state), and generalizability (skills for extending knowledge of specific interaction or behavior to similar situations in a new context). Two other usability attributes may indirectly affect learnability [48]: feedback (including observability and quality of content and its presentation) and robustness (including error prevention, recoverability, and provision of help).

Based on these approaches, ref. [49] suggested a framework for learnability evaluation comprising seven attributes: familiarity, consistency, predictability, informative feedback, error handling, and online help.

Familiarity is defined as the degree to which users’ prior knowledge and experience help them interact with a new system. This parameter depends on the ratio between the existing user’s knowledge and the knowledge required for effective interactions in the game. In the context of games, the concept of familiarity means that users can start playing quickly and easily determine how to initiate any interaction. Usually, this issue is reflected in the user interface design [50].

Consistency can be defined as related to the similarity of behavior resulting from similar situations or similar goals of the task. Consistent interfaces are easier to learn and use, thus helping users be more confident and inclined to try research learning strategies when using them [51].

Predictability concerns the support for the user to get to know the effect of future action, having in mind past interactions. In the game context, this user-centered concept is related to the player’s ability to determine the effect of actions and interactions within the game and reduce possible mistakes.

Informative feedback concerns the ability of the user interface to provide clear, comprehensive, and continuous reports of changes caused by a user’s actions. It also reflects the quality of presentation and content and receipt of an appropriate response from the game for each player’s actions.

Error handling refers to the help and guidance provided to the user, which enables identifying and correcting errors and maintaining the integrity of the game-playing process.

These attributes were applied with four contextual factors in mind: tasks (activities and assessments), learning environment (learning methodologies, teaching methods, and resources), user characteristics (gender, previous experience, attitude, and motivation), and learning tools. The seven attributes were addressed in a 16-item questionnaire for measuring the learnability of CASE tools in educational environments. Similarly, they may be used for measuring the learnability of educational video games, by applying game-based learning tasks, game-based learning environment, player characteristics as contextual factors, and educational games as a learning tool.

3. Research Methodology

3.1. Maze Game Creation

3.1.1. Design Methodology of Maze Game

The educational video game “Let’s save Venice” was designed and developed within the e-Creha Erasmus+ project [9]. The video game is a 3D educational maze enriched by 2D and 3D mini-games of various types [52] representing different didactic tasks. The game was designed by means of the serious game development platform APOGEE [51], for three months [20]. The game design was based on a formal, XML description of the enriched maze created in an incremental and iterative process and presenting the structure and the interior of the maze, together with the learning multimedia content and presentation and settings of the didactic mini-games. The XML description, together with game graphics and audio resources, were supplied to a custom plugin installed in the Unity 3D editor [52] and responsible for the generation of 3D mazes. The generated maze game lacks personalization of content and adaptation of task difficulty though such features are supported by the design process [53,54]. Next, some of the game objects (such as the hidden objects) were rearranged within the Unity 3D editor and several online versions of the maze game were built and deployed on the project Web server. Thus, the design of such 3D educational video mazes follows several consecutive steps:

• Setting learning objectives and creating a game scenario reflecting them;
• Collection and selection of relevant textual and multimedia learning content;
• Design of audiovisual layout of the maze halls;
• Selection of appropriate types of didactic mini-games and setting their location and content;
• Creation of an XML document representing the maze and the mini-games contained in the maze halls;
• Generation of the maze within the Unity 3D editor;
• If needed, game tuning in Unity 3D editor (e.g., rearranging the interior layout of some of the maze halls);
• Building online versions of the game and deploying them on a Web server; and
• Testing and validation of the pilot version of the game with target users.

The process of learning and playing should take a minimum time with maximum efficiency. To achieve the mission goals, the player must pass through all the maze halls, complete all mandatory mini-games, and find all hidden objects in the game by passing through all the maze halls and collecting the maximum number of points (i.e., achieve the maximal result). He or she can enter a new hall by unlocking the door leading to that hall, by giving the right answer to the door question concerning the didactic content placed on the learning boards in the current hall. Before answering the door-unlocking question, the player should play and solve all the mandatory puzzle mini-games in the current hall. After finding all the hidden objects, the player may opt between continuing the play, repeating the game, or finishing the play.

3.1.2. “Let Us Save Venice” Maze Game

As it is mentioned already, among the goals of the e-Creha project are providing innovative approaches to education regarding resilient architectural heritage and climate change, developing knowledge for building a culture of preparedness that addresses a challenge of climate change, and building a new climate-sensitive set of skills. That is why the game “Let’s save Venice” aims to explore the main factors, problems, and solutions for climate resilience and the protection of monumental cultural heritage in Venice. The target group of this game corresponds to the e-Creha project audience—students. Furthermore, it is focused on students of a wide range of university subjects, as the issues related to climate change are significant and multidisciplinary and impact society as a whole.

The structure of the game represents a five-hall maze with circular topology, as given in Figure 2. The maze map consists of the following halls: Introduction, The Context, The Problem, The Solution, and The Future. These halls are interconnected by locked doors that can be unlocked by the accurate answer to a didactic question. The arrows in Figure 2 show the direction of opening the respective doors. The game begins at an Introduction Hall describing the objectives of the game and the playing process. The player starts from the Introduction Hall, and after providing correct answers to the door unlocking questions, reaches the Solution Hall. There, he/she can move either to the Future Hall or to the Introduction or to return to the previous hall. After unlocking all the doors, the player can move through all the maze halls in any possible direction.

Figure 2. Topology of the maze.

All the halls have learning boards and different built-in mini-games (puzzles and quizzes) corresponding to the educational game scenario and learning objectives. Generally, the learning objectives are focused on exploring how built heritage is vulnerable to extreme weather and hazards and how these issues can be addressed most appropriately. Especially for this game, the goals are to experience the resilience strategies to protect the built heritage of the city of Venice against climate change, namely the floods. These objectives are achieved through different types of puzzle mini-games that provide didactic tasks dedicated to enhancing the player’s skills and knowledge. Therefore, all the maze halls present educational content corresponding to their name.

The player’s mission in the video maze game is to understand how Venice can be saved from ever-growing floods by getting acquainted with the multimedia materials on the information boards. Then, using the acquired knowledge to solve the built-in learning tasks (different types of mini-games). Currently, the mini-games built into the maze halls are the following five types [50]:

• Unlocking doors by answering a question—“Door unlock” mini-game (Figure 3a);
  

  Figure 3. (a) A screenshot of the “Door unlock” mini-game (Hall 1–4); (b) A screenshot of the “Rolling balls” (Hall 4); (c) A screenshot of the “Hidden objects” (Hall 2–5); (d) A screenshot of the “Word soup” (Hall 2); (e) A screenshot of the “Memory” (Hall 2); (f) A screenshot of the “Arrange me” (Hall 3).
• Rolling balls marked with text or a picture to the corresponding positions or rings on the floor map in the hall—“Rolling balls” mini-game (Figure 3b);
• Detection of didactic hidden (translucent) objects to obtain points and to read their description—“Hidden objects” mini-game (Figure 3c);
• Finding hidden words in a table with ordered letters by marking a sequence of letters—“Word soup” mini-game (Figure 3d);
• Finding hidden pairs of tiles (two identical words/images, or word and the corresponding image) by clicking on them—“Memory” mini-game (Figure 3e);
• Arranging images according to a given attribute by placing them in the corresponding positions—“Arrange me” mini-game (Figure 3f).

3.2. Evaluation Methodology

3.2.1. Survey Design

The goals of our survey design consisted of: (1) measuring both learnability and game experience, together with their most important contextual factors; (2) creation of a rather short and concise questionnaire (despite the complexity of dimensions of learnability and game experience) by asking players only one general question about each dimension, in order not to bore them with many questions requiring a long time for answers. For measuring game learnability, we decided to apply the Senapathi framework for learnability evaluation [47], by replacing the four original contextual factors with those that are specific for educational game validation, namely game-based learning tasks, game-based learning environment, player characteristics, and the climate resilience game as a learning tool. Therefore, in the first part of the survey, we asked the participants about their age, gender, the quantity of playing digital games of any kind (hours per week), and the quantity of playing digital games for learning. Furthermore, we asked for the most significant play outcomes, namely the final game score, the number of hidden didactic objects found in the game, and the total playing time. Based on previous research [55,56], we found that these three playing metrics serve as indicators of both learnability and playability of the game because their values demonstrate the overall player attitude towards the game, including interest, motivation, and engagement. The final game score is measured in points gained for solving mini-games in maze halls, where for each partial solution, the player receives a fixed number of points. For example, in the mini-game that requires players to find hidden objects and read the associated didactic contents, they receive 10 points per object found. The number of hidden objects in the maze halls is five. The maximal score for the whole game is 190 points (60 for mandatory and 130 for optional mini-games), where the player earns some points after successfully completing a task in a mini-game. The mean playing time was estimated at about 20 min based on some preliminary game tests with students at Sofia University as beta users. However, we expected large deviations from the mean value because the majority of the mini-games were not mandatory and, moreover, there was no obligation to find all the hidden objects and reach the final of the maze game. On the other hand, after reaching the end of the maze, there was no restriction to leave the game, so some more curious students returned back to the visited halls and continued playing with the mini-games.

In the second part of the survey, we used the Senapathi framework for learnability evaluation [49] by asking only one, general question about ease of learning, familiarity, consistency, predictability, informative feedback, error handling, and online help. Thus, we managed to address the six learnability dimensions by six complex items by bringing together all the items of the original questionnaire on a given dimension. Further, we followed the same approach for all the 33 items of GEQ [40] and designed a complex, general item for evaluating each component of game experience, namely flow, challenge, competence, positive and negative affect, immersion, and tension. Hence, we prepared six general items for game experience evaluation of the six types of mini-games integrated into the maze (presented in Section 4.1). The survey included seven contextual items for assessing user profile and playing results, six items for evaluation of learnability, and seven sentences for providing the opinion in a five-step Likert scale about the game experience playability for each mini-game. The questionnaire is presented in Appendix A at the end of the article. Further, we prepared and conducted a concise post-game semi-structured interview.

3.2.2. Semi-Structured Interviews

Together with the online survey aimed at quantitative data about game experience and learnability, we conceived a short semi-structured interview as an instrument for qualitative research. The goal was to collect some open-ended data regarding the students as players and their impressions about the game learning content and its playability, by means of several open-ended questions asked during an informal dialogue between the researcher and the player. The qualitative data received from such an interview were expected to reveal some subjective concerns, feelings, and beliefs of the participants about the educational video game and its gameplay and then, to be applied for improvement of the design of the future three games planned to be created within the e-Creha project. The semi-structured interviews were planned to be conducted with some of the players after the game session and after answering the questions of the post-game online survey.

3.2.3. Workshop Setup

The testing and validation of the pilot version of the game “Let’s save Venice” was performed during the first e-Creha project workshop named “Climate Xtremes and Resilient Heritage.” The student workshop was held from 31 of August to 8 of September 2021 in Technical University at Eindhoven, the Netherlands, and was dedicated to implementing and testing new pedagogies according to the specific themes of the modules regarding climate resilience of monumental cultural heritage. The participants were students, professors, and researchers from five different universities that focus on climate resilient solutions for a cultural landscape area. The Sofia University team is one of the workshop co-organizers, and therefore it shares the responsibility of the workshop program. The project partners support us in all dissemination activities. Therefore, publishing the results about game validation will contribute to a better understanding and enlarging of the impact of the project outcomes.

4. Experimental Results

4.1. Player Demography

In total, 24 participants (12 males, 12 females) took part in the pilot game evaluation. All of them were either graduates (21 students) or Ph.D. students (3) in the e-Creha partners’ organizations, namely from the Technical University of Eindhoven, TOBB University (Turkey), Sofia University (Bulgaria), and University of Molise (Italy). Before the game session, they did specific environmental tasks and developed some additional environmental competencies and awareness as part of the workshop (https://www.ecreha.org/student-workshops (accessed on 29 October 2021)). The mean age was 25.20 with a standard deviation of 6.48. Most of the students do not play digital games, as 62.5% of the participants admit to playing between 0–1 h per week, 25% 1–5 h per week, 1 student 5–10 h per week, and only 2 (8%) play digital games more than 10 h per week. Approximately 87.5% of respondents do not play digital games for learning or training purposes, and only 12% (3 of them) play 1–5 h per week.

4.2. Evaluation of Learnability

The participants in the workshop (N = 24) achieved game scores between 27 and 190 points. Only three of them succeeded in gaining the maximal score of 190 points. The average score was M = 131.45455 points, with SD = 54.89707 and SE = 11.20582. On the other hand, seven of them found all five didactic hidden objects, while the other two found two objects only. The descriptive statistics for the hidden objects mini-game are M = 3.75, SD = 1.48177, and SE = 0.30247, while for the playing time they are M = 23.25 min, SD = 15.82032, and SE = 3.22931.

The average playing time (Figure 4) was planned to be 20 min but two players finished the game in less than 5 min, while another played 65 min. The differences and the deviation in playing time are because of the fact we did not ask the students to play all the non-mandatory mini-games and to find all the hidden objects. For these three metrics of game outcomes, the statistically significant medium correlation was found only between the game score and the number of found hidden objects (r = 0.50782, p < 0.05), which indicates that the number of points depends on the found objects, but also on the results of other mini-games.

Figure 4. Playing time distribution (in minutes).

Figure 5 presents the survey results concerning the distribution of respondents’ responses on the Likert five-point scale for each game’s learnability attribute. The results are given in percentages. It can be noted that positive attitudes of players prevail—agree and strongly agree opinions are more than 50% for each attribute of learnability. The detailed descriptive statistics of these attributes, shown in Table 2, confirmed this observation.

Figure 5. Survey results for the attributes of game learnability.

Table 2. Descriptive statistics of the attributes of game learnability.

	Ease of Learning	Familiarity	Consistency	Predictability	Informative Feedback	Error Handling
M	3.45833	3.91667	3.62500	3.83333	3.41667	3.45833
SD	1.35066	1.17646	1.13492	0.96309	0.97431	1.06237
SE	0.27570	0.24014	0.23166	0.19659	0.19888	0.21685

Pearson correlations between the six components (attributes) of learnability are presented in Table 3, where statistically significant correlations are given in bold with a level of significance coded by one, two, or three asterisks and having a p-value less than 0.05, 0.01, and 0.001, correspondingly. We found medium and strong correlations between all the components excluding error handling and the correlation between predictability and informative feedback error handling appeared with a statistically significant correlation only with predictability.

Table 3. Pearson correlations between the six components of learnability.

	Ease of Learning	Familiarity	Consistency	Predictability	Informative Feedback	Error Handling
Ease of Learning	1
Familiarity	** 0.59969	1
Consistency	*** 0.71264	** 0.52916	1
Predictability	** 0.52922	* 0.48606	* 0.41767	1
Informative Feedback	** 0.60847	* 0.48679	** 0.57997	0.30890	1
Error Handling	0.21084	0.34497	0.18481	* 0.41786	0.35354	1

* p < 0.05; ** p < 0.01; *** p < 0.001 Statistically significant values are in bold.

4.3. Game Experience of the Mini-Games

The mean values of the experience factors for the mini-games (Figure 6) discover that using the Likert scale (min 1—strongly disagree, max 5—strongly agree, 3—neutral), all mini-games got comparable scores. The highest results (average mean of 3.59) were observed for both factors: immersion (the game content was interesting and well presented, which encouraged me to explore) and positive affect (I liked the game and had fun). In third place with a mean value of 3.53 was the factor of competence (I felt skillful and successful in the game and quickly achieved the game’s goals). The high values of immersion, positive affect, competence, and flow show clearly that the game brings a great experience to its players. At the same time, the challenge and tension factors showed moderate values for the majority of the mini-games, which proves the game was not rather challenging for the students. As a whole, students found the maze game and the integrated mini-games integrated to be not quite a gameplay challenge. On the other hand, the results revealed that the level of immersion and competence were relatively high (see the questions in the Appendix A for their way of determining). This fact does not make any logic discrepancy as far as high immersion is provoked by a well-designed maze environment together with appropriately selected audio/visual effects, music, etc. Further, high competence is achieved by intriguing learning contents and engaging didactic tasks.

Figure 6. Mean values of the experience factors for all mini-games.

The factors tension (I felt tense and was discouraged by the game), challenge (the game was difficult and required effort, which made me spend more time), and negative affect (playing the game was tiresome and I felt bored and distracted) were evaluated negatively (Table 4).

Table 4. Descriptive statistics of the experience factors.

	Flow	Challenge	Competence	Positive Affect	Negative Affect	Immersion	Tension
M	3.3687	2.8079	3.5366	3.5893	2.5019	3.5929	2.2999
SD	0.2459	0.3466	0.2109	0.1602	0.1825	0.0708	0.2802
SE	0.05019	0.07074	0.04305	0.03269	0.03726	0.01446	0.05719

Table 5 presents the Pearson correlations between the seven components of the game experience for the word soup. For the other games, the correlations appeared to be similar but slightly less expressed. Medium and strong correlations were found between almost all the components. Negative correlations were expected because of the opposite nature of the components in the respective pair.

Table 5. Pearson correlations between the seven components of the game experience for the word soup mini-game.

	Flow	Challenge	Competence	Positive Affect	Negative Affect	Immersion	Tension
Flow	1
Challenge	−0.13608	1
Competence	** 0.58670	** −0.55452	1
Positive Affect	** 0.61133	** −0.60586	*** 0.73469	1
Negative Affect	−0.25454	** 0.64661	** −0.59075	** −0.62975	1
Immersion	** 0.54244	** −0.56286	*** 0.69888	*** 0.75805	*** −0.70485	1
Tension	−0.24561	** 0.63769	** −0.54182	** −0.55554	*** 0.67440	** −0.53815	1

** p < 0.01; *** p < 0.001 Statistically significant values are in bold.

Table 6 provides Pearson correlations between competence and the other components of the game experience for all the mini-games. The strongest correlations were between competence and all the other components.

Table 6. Pearson correlations between competence and the other components of the game experience for all the mini-games.

	Door Unlock	Rollingballs	Hidden Objects	Word Soup	Memory	Arrange Me
Flow	0.38623	*** 0.69717	0.27812	** 0.58670	0.40585	* 0.50176
Challenge	** −0.63700	* −0.50415	** −0.61585	** −0.55452	−0.33300	−0.35172
Positive Affect	*** 0.74074	*** 0.78106	** 0.60170	*** 0.73469	** 0.60302	* 0.51785
Negative Affect	** −0.55954	*** −0.81833	** −0.59134	** −0.59075	** −0.56350	* −0.42204
Immersion	0.35626	** 0.56717	* 0.48537	*** 0.69888	** 0.61961	** 0.56554
Tension	* −0.43041	*** −0.70670	−0.28157	** −0.54182	** −0.59899	** −0.52188

* p < 0.05; ** p < 0.01; *** p < 0.001 Statistically significant values are in bold.

Finally, we calculated the T-test for mean values of the seven components of the game experience for each one of the mini-games. Statistically significant differences in means were found only for flow, challenge, and tension.

4.4. Results of Semi-Structured Interviews

After the game sessions and the online survey, we conducted semi-structured interviewing with seven of the game players. The questions were asked in an informal dialogue with the students and altogether took about ten minutes. All the interviewed students appeared to be passive game players, with some experience mainly in 2D casual video games. They shared with us that they were impressed by the maze game both during the pre-game presentation and video explaining how to play and, as well, in the game session that followed. All the respondents highly appreciated the learning content about Venice, the local floods, the consequences arising from climate change, and the MOSE project for preventing the floods. They liked the content layout in the maze and in the mini-games stating that such puzzles make the maze game much more interesting and appealing. Three of the respondents proposed to design for the next version of the maze some mini-games with levels and, therefore, to represent more learning content structured by its complexity level. In addition, the respondents appreciated the game mechanic of unlocking doors by answering a question but proposed to include a quiz mini-game in the final hall of the maze, which will contain questions about all the didactic content presented in the maze.

Together with the positive feedback, we received some critiques about the game design. Two of the respondents complained about the mouse interface for navigating through the maze—they found it too tricky and complicated. All of the respondents mentioned the lack of adequate help for the “Arrange me” 3D mini-game. Some of them proposed ideas for including pop-up windows or similar mechanisms for showing contextual help for any 3D mini-game. One of them was bored by the music records reproduced in the maze halls and proposed to include a button for muting only the music but not the sound played for game events.

5. Discussion

The outcomes of the game experience evaluation prove that the educational online maze game “Let us save Venice” responds successfully to the learnability and UX criteria, as defined in Section 2.4. The learning outcomes, measured by game points, hidden objects collected, and time for playing proves that players, most of whom are not used to playing games, succeeded in completing the game learning objectives.

The analyses of both the survey and semi-structured interviews provide an answer to the research question stated in the introduction: immersion, positive affect, and competence appeared to be the main experience factors contributing to raising awareness of climate resilience and built heritage sustainability. As Table 5 reveals, competence correlates highly with other components of the game experience for all the mini-games, especially with positive and negative affect. The authors obtained similar results for immersion, which proves the high interrelationship between all the experience factors. On the other hand, the correlations were higher for the “Rolling balls” and “Word soup” mini-games, while all the mini-games demonstrated high levels for immersion, positive affect, and competence.

Considering the main factors for learnability, the factor ease of learning strongly correlated (r = 0.71264, p < 0.001) with consistency and informative feedback. The result proves the positive correlation between good game design and the learnability of serious games. These factors are defined by the following sentences in the questionnaire (Appendix A):

• ease of learning—It was easy for me to learn the game’s rules without looking for additional help and to start playing and learning new things from the information boards in the maze halls.
• consistency—It was easy for me to go through the maze and perform game tasks (puzzles and mini-games).
• informative feedback—I got an appropriate response from the game for each one of my actions.

According to players’ feedback, all mini-games prove to contribute to the positive gaming experience. The mini-games “Word soup”, “Memory”, and “Arrange me” collected the highest mean scores for factors positive affect and immersion. The mini-game rolling ball was ranked highest for the factor tension. The factor of challenge (the game was difficult and required effort, which made me spend more time) showed the highest standard deviation (SD = 0.3466).

More specifically, the outcomes of the video maze game “Let us save Venice” and the mini-games analyses prove that emphasizing user-centered game design plays a significant role in creating immersive and enjoyable educational video maze games [34]. Furthermore, game evaluation results highlight that user experience and game playability have to be specifically addressed in the design phase as they contribute to learnability and raising awareness. Following our analysis, some more general considerations about raising awareness in sustainable development, climate resilience, and built heritage protection issues are presented:

• Target auditory: The perception, attitude, and knowledge of sustainability and environmental issues of university students are of great importance in contemporary society [57]. The involvement of young people in sustainability decision-making is critical to the development of long-term policies. Further, students are a significant part of society and a driving force for changes in attention to environmental issues, as they usually are the leaders in modern ecological campaigns. Climate awareness, resilience strategies, and approaches need to become a horizontal competence for a large group of experts in different fields and citizens as a whole.
• Problem domain: Climate change and climate resilience need to be addressed as an interdisciplinary issue and thus combine the efforts of different experts.
• Institutional role: Higher education and universities play a fundamental role in training professionals who will have a crucial impact in preserving the environment in the near future. Hence, students need to be aware of sustainability and environmental issues, thus helping establish better pro-sustainability attitudes and behavior in the future generation.
• Serious game role: It is well known that one of the most efficient approaches to reach young minds is through video games, so we have developed an educational maze game aimed at raising awareness of sustainability issues in the context of Venice. During the semi-structured interviews, students’ answers revealed that even a game-playing session of half an hour could raise awareness about climate resilience and the preservation of cultural monuments. In this manner, with this game, we try to encourage the adoption of a pro-sustainability attitude among students. This attitude is critical for reducing the human impact on the environment and reaching more sustainable development in the future.

The serious game complements the e-Creha learning activities and other initiatives for building new competencies and raising awareness about climate resilience, sustainability, and preservation of cultural monuments, such as:

(1).

Promotional materials such as the project website, leaflets, videos, and others focused on the general public;

(2).

Dedicated workshops with lectures, video materials, software modeling and simulations, field studies, discussions, and practical projects; and

(3).

Multiplayer events ‘Building better Climate Heritage Europe’ consisting of presentations and open discussions between all participants.

6. Conclusions

The article presented a study exploring the area of educational video games where the user-centered approach was applied to the design of a 3D maze game focused on the climate resilience of monumental cultural heritage. The online educational game “Let us save Venice” is a video maze enhanced with various types of mini-games. It is dedicated to one of the global sustainability problems—ever-growing floods and their impact on the monumental built cultural heritage of Venice. The game is created by using the user-centered approach that considers the process of design and development according to the needs and characteristics of the target users, i.e., players (learners). The paper presents and analyzes the results of pilot game validation through playing sessions, questionnaires, and structured interviews. The validation process proved our three main expectations:

• All the learnability factors were highly appreciated, especially familiarity, consistency, and predictability. The results proved that familiarity, consistency, predictability, and informative feedback contribute essentially to ease of learning, based on the found significant strong correlations between them;
• All the considered factors of user experience—flow, challenge, competence, positive and negative affect, immersion, and tension—are significant, especially immersion, positive affect, and competence, whose mean values for all the mini-games exceeded 3.5. The factors of flow and tension received mean values lower than 2.8 and 2.5, respectively, which brings the idea of improving the difficulty of the mini-games, probably with an appropriate game level design;
• According to the feedback of players obtained through the online quiz and semi-structured interviews, all mini-games proved to contribute to a positive gaming experience. In particular, “Rolling balls” and “Arrange me” reflect factors challenge and tension, “Word soup”—competence and positive affect, and “Hidden objects” and “Memory”—flow. The players reported being moderately competent in playing the mini-games. The competence factor appeared to be highly positively correlated to flow, positive affect, and immersion, and negatively correlated to challenge, negative affect, and tension. Moreover, the players shared with us that the benefit of such a short game playing session is mainly in raising awareness about the problem domain, and subsequently, they would like to return to the game and play it longer.

The next version of the pilot game “Let us save Venice” should integrate the recommendations and findings observed in the analysis phase. Appropriate difficulty levels will be added to all the mini-games, with adjustments to the level of difficulty at a given level being done in a dynamic way based on the player outcomes. Moreover, the new versions of the maze will contain new types of mini-games such as quizzes and classification mini-games, which are currently under construction. On the other hand, the future work should incorporate the new functionality of the APOGEE platform, allowing the game to adapt and adjust to the players’ playing and learning style, emotional state, or knowledge on the subject [58]. Further functions can include a non-player character (NPC), who might assist players while coping with challenges and in providing more insights and support.