Development and Evaluation of a Game to Foster Sustainable Self-Help and Mutual Help Education for Disaster Prevention

Abstract

This study explores the development and evaluation of a game aimed at fostering sustainable self-help and mutual help education for disaster prevention. The game, developed using Unity and Blender, addresses the critical need for effective disaster preparedness, emphasizing the importance of community cooperation, as evidenced by the Great Hanshin-Awaji Earthquake, in which most rescues were performed by neighbors. Additionally, it features realistic disaster scenarios, and the game’s design incorporates gamification and simulation elements to enhance learning and engagement. An experiment involving 20 participants aged 20–21 years was conducted to evaluate the game’s effectiveness. Participants played the game on desktop personal computers for at least 10 min, and their performance and awareness were measured through pre- and post-gameplay questionnaires. The results indicated no statistically significant improvement in the ability to cooperate with strangers, provide correct instructions, or overall consciousness of helping others. However, a slight increase in the average scores was observed. Participant feedback highlighted the game’s realistic approach and suggested improvements in operability and platform compatibility. The study concludes that while the game shows promise, further development and research are needed to enhance its educational impact and effectiveness in disaster preparedness.

1. Introduction

Kawada reported that 77.1% of those rescued during the Great Hanshin-Awaji Earthquake that occurred on 17 January 1995 were rescued by neighbors, and only 22.9% were rescued by the fire department, police, and Self-Defense Forces [1]. Morisaki et al. found that the rescue rate by self-help and mutual aid was high, partly because the government could not sufficiently support victims owing to the paralysis of administrative functions caused by the earthquake, confirming the importance of self-help and mutual aid [2].

Mutual aid is defined as cooperation with neighbors and local residents [3]. It is crucial to increase the sense of mutual aid during disasters, in addition to promoting self-help measures. Therefore, it is important to undertake educational initiatives and other efforts to enhance mutual aid capabilities. However, upon reviewing previous research, it appears that while there are many initiatives focused on enhancing self-help capabilities, there are relatively few aimed at improving mutual aid capabilities. From the perspective of engineering for education, previous approaches have helped improve the sense of self-help. For example, Matsuno et al. developed a smartphone application based on gamification to simulate flood disaster evacuation [4]. Arakawa et al. developed a smartphone and tablet application based on gamification for tsunami disasters [5]. In some cases, learning media supported by the use of smartphones have been developed to increase literacy about volcanic eruption disaster mitigation or flood disasters [6,7]. Itamiya developed Virtual Reality (VR) and Augmented Reality (AR) content for evacuation drills following tsunamis and earthquakes [8]. However, the applications of these technologies in engineering for education have been limited to self-help.

In contrast, many educational initiatives exist for mutual aid, such as workshops [9,10]. However, unlike self-help, only a few previous studies have attempted to raise awareness of mutual aid during disasters through instruments. To the best of our knowledge, the only case in which mutual aid can be studied using smartphones was that of Chien et al. [11]. To provide sustainable disaster education and firmly establish awareness so that people can prepare for disasters, it is important to educate individuals regarding both self-help and mutual help simultaneously—and to do so in an easy and continuous manner using instruments. Therefore, if technologies for mutual aid are developed, mutual aid capacity might be expected to improve. Mayer showed that multimedia learning leads to meaningful learning [12].

In addition, in many local communities, residents’ awareness of the need for self-help and mutual aid is limited to “direct experience of disasters” and “information from mass media and hearsay”. Consequently, many local governments do not provide residents with opportunities to acquire specific knowledge and skills for disaster management [13]. Thus, local resident participation has been sluggish, although many local governments provide residents with opportunities to acquire specific knowledge and skills for disaster management. Additionally, owing to the fading of disaster experiences and natural and social changes in the population, awareness of the need for self-help and mutual aid has decreased over time, even in areas previously affected by large-scale disasters. To ensure the sustainability of disaster reduction efforts within local communities, it is necessary to provide residents with regular opportunities to become aware of their need for self-help and mutual aid [13].

Therefore, it is important and meaningful to develop engineering technologies that contribute to education aimed at enhancing not only self-help capabilities but also mutual aid capabilities. Simulation and gamification are recognized as effective methods for gaining “awareness” of events that are difficult to experience in reality, such as disasters [13]. According to Kapp, gamification is defined as “using game-based mechanics, aesthetics, and game thinking to engage people, motivate action, promote learning, and solve problems” [14]. Moreover, gamification integrates game elements and thinking into nongame activities [15]. Therefore, introducing gamification into disaster evacuation education would be useful. Previous studies have applied gamification in disaster education. For example, the card game “Crossroad” involves participants viewing scenarios on cards as their own problems, exchanging opinions, and indicating their thoughts with “yes” or “no” answers [16]. An online game, “Stop Disasters”, teaches both adults and children on building safer villages and cities to withstand disasters. Through gameplay, participants learn how the location and construction materials of houses make a difference when disasters strike, and how early warning systems, evacuation plans, and education save lives [17]. “LIFE” is an interactive board game, and it aims to overcome disasters without anyone dropping out and to learn self-help, mutual aid, and public help skills by playing the game together with other people in the room, creating a sense of dialogue and unity [18]. Educational tools that apply gamification are available, but they typically apply gamification alone. As mentioned at the beginning of this paper, developing tools that combine simulation and gamification may provide further educational benefits. Deterding et al. pointed out that gamified elements can boost learners’ motivation, making it an effective tool for educational purposes [19]. Additionally, social learning theory suggests that social behaviors can be learned through observation and imitation [20], implying that cooperative behaviors in the game could transfer to real-world actions. Additionally, from the perspective of self-efficacy, successful experiences in the game can enhance players’ confidence in their ability to perform self-help and mutual aid during disasters [21]. To the best of our knowledge and based on our survey, no study has theoretically discussed how digital games improve self-help and mutual aid capabilities, nor how these games transfer to real-world behaviors. However, based on previous studies, we hypothesized that by observing and imitating the actions of characters in the game who prepare for disasters and help others, players can enhance their self-help and mutual aid capabilities.

This paper presents a case study on developing a game for educational purposes aimed at raising awareness of self-help and mutual aid in the event of a disaster, especially a fire. The game includes events that check and improve players’ awareness of self-help and mutual aid. In addition to developing the game, the effects of playing games and increasing the awareness of self-help and mutual aid were evaluated using a questionnaire survey. In the previous paragraph, we explained that simulation and gamification are recognized as effective methods for gaining “awareness” of events that are difficult to experience in reality. We believe that the application of simulations and gamification in disaster education will have other effects. Advancements in declining birthrates and aging populations have been observed in several countries [22]. Additionally, digital communication is related to lower social connectedness [23]. Concerns have been raised about the declining participation of young people in local disaster management organizations, which may become weaker in the future [24]. One reason for this may be the lack of face-to-face communication, which may be an obstacle to enhancing mutual aid. Therefore, it is necessary to consider methods for increasing mutual aid, as well as the membership in organizations and face-to-face communication.

The significance of this study can be summarized as follows:

• The development of a game for increasing the awareness of self-help and mutual aid during a disaster.
• Evaluation of the effect of a game and whether awareness of self-help and mutual aid increased.

The remainder of this paper is organized as follows. Section 2 describes the game’s development. Section 3 describes the experiment conducted to evaluate the effects of the game. Section 4 presents the experimental results. Section 4 presents the considerations derived from the experiment’s results. Section 5 provides a discussion, including the limitations of this study. Section 6 concludes the paper.

2. Design and Development of the Game

2.1. Design of the Game

2.1.1. Tools Used for Development

The game was created using Unity ver. 2020.3.18f1 (by Unity Technologies, San Francisco, CA, USA) [25], Blender ver. 3.4 (by Blender Foundation, Amsterdam, The Netherlands), and C# ver. 8.0. Unity was selected because of the abundance of information available for games and asset development, whereas Blender was chosen because it facilitated the creation of objects.

2.1.2. Concept of the Game

The game was developed based on the following concepts:

• The player must rescue an injured person outside.
• The player must successfully evacuate people who are panicking outdoors.
• In addition to helping others, the player must also evacuate themselves quickly.

To enhance realism, the game was designed as an action game that allows players to move freely in a three-dimensional (3D) space. Collines showed that a distinguishing feature of video games is their interactivity, and that sound plays an important role [26]. According to this, music was created to evoke a sense of urgency and heighten the tension of the situation. To promote communication with others, a mechanism was implemented that allowed players to interact with other characters when they stopped. To further promote an understanding of the power of mutual help, we incorporated a system in which multiple options are displayed when a player speaks to others, and the score increases as the player gives correct instructions. This system is expected to help players understand the meaning of mutual help while realizing the importance of cooperation and effective communication with others. Gentile et al. demonstrated that games in which characters collaborate and support each other nonviolently can enhance both short- and long-term prosocial behaviors [27]. While this previous study does not specifically address mutual aid, based on this previous study, our game has the potential to promote communication for social support among players.

Figure 1 presents a brief flow of the game. The game’s details are explained in the following section.

2.2. Development of the Game

2.2.1. Purpose of the Game

The game was created to enhance self-help and mutual help skills by providing players with disaster prevention knowledge and fun. Players earn points by rescuing people from the fire scene, cooperating with others while communicating with them to carry items such as packages, as well as quickly evacuating themselves. The game also features a scoring system in which players compete to achieve a high score.

2.2.2. Stage Configuration

The game comprises two stages: an “Inside a dream” stage and an “Evacuation” stage. The following describes the concepts of each stage.

The “Inside a dream” stage serves as an introductory phase, allowing players to navigate their characters through picturesque green meadows (Figure 2). As time progresses, flames gradually emerge and spread across the area. The game is initiated when the player becomes enveloped by the advancing flames. This stage is designed to evoke the sensation of experiencing a fire within a dream, rather than simulating a live fire scenario.

The “Evacuation” stage is the main stage of the game. Following the ”Inside a dream” stage, players awaken to find themselves in a house that is ablaze, with injured or incapacitated individuals inside. In this stage, players must carry and rescue these individuals (see Figure 3a), transport them to a safe location outside the house (see Figure 3b), and escape from the building within a specified time limit.

Outside the house, many people are disoriented because of the unexpected fire (see Figure 3c). To address the situation effectively, players must enlist the assistance of others, as managing the crisis alone is impractical. Players are required to direct these confused individuals, for example, by instructing them to call for help (see Figure 3d) or by guiding them to evacuate. This necessitates careful consideration of the type of assistance to request, emphasizing the game’s core objectives of learning about “self-help” and “mutual help”.

The game ends when the player finishes the evacuation or when the time limit is reached.

2.2.3. Elements to Improve Realism of the Game

The following elements were incorporated to improve the realism of the game:

(1)

“Fire” and “smoke” gauges:

In an actual fire scene, it is crucial to pay attention to fire and smoke. Individuals must avoid exposure to flames and smoke. Therefore, two kinds of gauges were displayed at the top right of the screen: “fire” and “smoke” gauges. Both gauges increase gradually when the player remains in the house. Additionally, these gauges increase rapidly if the player touches the fire and smoke. Therefore, the player must navigate the environment without coming into contact with flames or smoke, particularly by crouching to minimize exposure. The game concludes when either of the gauges reaches its maximum capacity. Figure 4 shows the “fire” and “smoke” gauges.

(2)

Scoring system:

To enhance the gameplay from the point of gamification, the player receives a score if they behave or direct others correctly.

Table 1. Player actions and points.

Behavior	Points to Be Added
Carry out injured or fallen people	1500
Give the right instructions	1000
Pick up or receive items	1000, 500, 100 (depending on item)
Call an ambulance or fire truck	Total score × 2.0
Could not evacuate	Total score × 0.5

details the actions that contribute to earning points. As shown in Table 1, actions directly related to saving people’s lives yield more points. In addition, players are also required to successfully evacuate themselves. In this study, we did not analyze the score of each participant.

(3)

Choice of behavior:

As shown in Figure 3a,b, the player can rescue people by carrying them out of the fire scene. However, when the player chooses to carry a person, the game’s pace slows down, and the smoke is inhaled, thus leaving the decision to rescue the victim up to the player’s discretion.

(4)

Set time limit:

A time limit is set, and points are awarded for rescues completed within this timeframe. In this game, the limit was set to 300 s from the start. Once this period expires, the game is over. Therefore, players must execute rescues and make decisions promptly. The remaining time is displayed in the upper left corner of the screen, as shown in Figure 4.

3. Experiment

3.1. Participants

An experiment was conducted to evaluate the effect of the developed game’s effectiveness. Twenty 20- and 21-year-olds participated in this experiment.

3.2. Experimental Procedure

The participants used desktop personal computers (PCs), and the end condition was to play for at least 10 min or until the player reached 10,000 points. If the player was able to rescue the victim within the time limit, the remaining time × 10 points were added. Data and feedback obtained from questionnaires before and after the participants played the game were collected and analyzed to determine improvements in their ability to help each other. Additionally, the game was evaluated.

3.3. Data Extraction and Analysis

This study conducted a questionnaire survey focusing on three different content areas to clarify the effects of pre- and post-game play activities on the ability to help each other. The survey items were as follows:

Q1

Were you able to cooperate with strangers?

Q2

Did you give the right instructions?

Q3

Are you conscious of helping others?

In addition to these three questions, free responses regarding impressions of the game were provided. These questions were used to determine the effectiveness of the game in developing mutual help skills and whether the players’ awareness of mutual help skills changed throughout the game. It is reported that the burden increases or decreases as the number of questions increases or decreases, respectively, and, the longer the survey, the greater the burden on the participant [28]. Therefore, we minimized the number of questionnaires and included only the essential items. The questionnaire items were not based on any previous studies, as no prior research has evaluated self-help or mutual aid. Consequently, we developed the items independently without referencing previous studies. The questionnaires in Q1 and Q2 were designed to have respondents select phrases used in the game, such as calling for an ambulance or requesting others to evacuate. These questions were created to evaluate the effectiveness of cooperative calls in the game context. Q3 assessed the effectiveness of the game event, in which participants were tasked with rescuing someone who had fallen inside a house.

Each item was rated on a 5-point Likert scale: 5 (very true), 3 (neither true nor false), and 1 (not true at all).

4. Results and Considerations

4.1. Results of the Questionnaire

The questionnaire was used to examine whether the game had any effects. In this study, the same questions were asked before and after gameplay and compared to verify the game’s effects. Figure 5 shows the box and whisker plots for Q1, Q2, and Q3. Figure 5a–c show the results for Q1, Q2, and Q3, respectively.

The average scores before and after gameplay for Q1 were 4.05 and 4.00, respectively. The Wilcoxon signed-rank test revealed no significant differences between the scores before and after playing the game (p = 0.8127). Therefore, the consciousness of cooperating with strangers did not statistically improve by playing games.

Next, the average scores before and after gameplay for Q2 were 3.70 and 3.75, respectively. The Wilcoxon signed-rank test revealed no significant differences between the scores before and after playing the game (p = 0.9148). Therefore, playing the game did not statistically improve the capability to provide correct instructions.

Finally, the average scores before and after gameplay for Q3 were 4.1 and 4.24, respectively. The Wilcoxon signed-rank test revealed no significant differences between the scores before and after playing the game (p = 0.627). Therefore, the consciousness of helping others did not statistically improve by playing games.

4.1.1. Cooperation with Strangers

From the Q1 results, no significant difference was observed in cooperation with strangers before and after playing the game. In addition, the average scores before and after gameplay were nearly identical. Therefore, we can say that the game had no effect on improving the awareness of cooperation with strangers.

There may be several reasons for the lack of improvement in cooperation with strangers when playing games. Jilson et al. showed that an effective disaster response requires robust communication and coordination among all stakeholders, including NGOs, local authorities, and the community [29]. Another report indicated the necessity of building partnerships with the government, universities, and the private sector. It also emphasized the full utilization of local parks, community facilities, grant programs, and workshop tools to enhance mutual aid, among other resources [30]. Therefore, simply communicating with others and giving directions in a game is insufficient for developing the ability to cooperate with strangers. For example, game-playing efforts should be established as a framework for improving mutual help capabilities in cooperation with communities, local governments, and businesses. In addition, the continuous implementation of gameplay by residents is expected to enhance their ability to cooperate with strangers. This suggests that increasing the awareness of cooperation with strangers requires the help of communities, local governments, and businesses.

4.1.2. Giving the Right Instructions

From the Q2 results, no significant differences were observed in giving the correct instructions before and after gameplay. Therefore, it can be said that the participants could not improve their awareness of providing instructions to others. We posit that the lack of a significant difference is due to the limited number of options provided for directing individuals outside the house. Additionally, it is possible that the simplicity of these questions contributed to the results. For example, Figure 3d illustrates two options for directing the person outside the house: “Please inform the surrounding houses about the fire”, and “Please shout ‘Fire!’ and act wildly here”. In the development game, incorrect answers were included in the choices because the game was designed to be enjoyed by everyone, including children. While the game’s content may indeed have been accessible, we cannot expect to improve the awareness of giving the right instructions without making the options more difficult.

4.1.3. Consciousness of Helping Others

From the Q3 results, no significant differences were observed in the awareness of helping others before and after playing the game. However, the average score after playing the game increased by 0.14 points. Therefore, although no significant differences were observed, the game may improve the consciousness of helping others. The game may have increased the consciousness of helping others, as its content predominantly revolves around helping others. This includes scenarios such as aiding an injured person or helping others with assistance from people outside the house. However, we believe that there are two reasons why no statistically significant effects were observed from playing the game. First, the experiment was conducted over a brief period, which may have prevented the participants from fully developing a sustained awareness of altruism. Second, the game featured only a limited range of scenarios involving helping others—specifically, assisting an injured or fallen individual and providing correct instructions to someone outside—which may have restricted the potential for significant behavioral effects. Other situations include inducing evacuation and helping the disabled and elderly. Therefore, we propose that designing a game that satisfies these two requirements will enhance its effectiveness.

4.2. Free Responses

Next, we discuss the free responses from participants. The results of the participants’ free responses are presented below. Note that not all participants responded to the free responses, which included the following:

• It was difficult to operate.
• It was really great!! The scenario of disasters, such as poor visibility, felt very realistic and well done.
• It was fun.
• It was interesting!
• The game had a lot to teach, and it was fun, but it was difficult because I’m not used to using a PC. I hope there is a smartphone version!
• The details like crouching to avoid smoke and vending machines having addresses were nice touches. Brilliant!
• Amazing!
• It was really interesting.
• There were times when I did not know what to do, so the tutorial could use some improvements.
• I scored 60,000 points! I will play again.
• As someone without game development knowledge, I felt this was really well made. At this level of quality, it would not be out of place on Steam.
• Lifting people was difficult.
• I want to become a firefighter!
• I genuinely think it is amazing that you can create such a game. I would be honored to help in any way I can. Keep up the good work!
• Make the color scheme easier to understand.

In summary, the free responses can be broadly categorized into two main themes: the quality of the game itself and the educational effectiveness of the game in enhancing self-help and mutual assistance capabilities.

Nearly all the participants who responded to the free-response section agreed that the game had a realistic feel. One participant commented, “It was really great!! The scenario of disasters, such as poor visibility, felt very realistic and well-done”. Another mentioned, “The details like crouching to avoid smoke and vending machines having addresses were nice touches. Brilliant!” These positive responses highlight the game’s attention to detail in pursuing realism. However, no comments addressed how the game changed their awareness of helping others. Given the previous observations, it is unlikely that the participants would perceive the game as improving their awareness of helping each other after only about 10 min of gameplay, despite its ease of use.

Additionally, the use of a PC platform for the game was noted to have somewhat reduced the ease of playing. One participant stated, “The game had a lot to teach, and it was fun, but it was difficult because I’m not used to using a PC. I hope there’s a smartphone version!” To facilitate broader enjoyment, it would be beneficial to port the game to a smartphone or home video game console, for example, by utilizing programming tools such as ‘petit-com 4 smile basic’ for the Nintendo Switch [31].

5. Discussion

We developed a game to increase awareness of self-help and mutual aid during disasters, and we evaluated the effects of the game and whether the awareness of self-help and mutual aid increased. The game contents were designed based on previous studies [26,27]. Based on observations of the gameplay, it was confirmed that all participants enjoyed the game. However, according to the results of the experiment, we could not confirm that the game had an effect on improving the awareness of cooperation with strangers or giving instructions to others. As discussed in the previous section, the participants engaged in a brief game to evaluate their awareness of mutual assistance. It was suggested that a duration of only 10 min would be inadequate for assessing the game’s effectiveness. It is important to consider an appropriate timeframe for a comprehensive evaluation, because a single day of experimentation may prove insufficient. Extended periods are required for a thorough assessment of the game’s impact. The experiment also faced a bias in participant age, as only those in their 20s, who were generally more familiar with games, were included. However, it is necessary to conduct experiments with a wider age group. By verifying the game’s effects across diverse age groups and genders, it can be promoted as a sustainable mutual help education method.

Conversely, although no significant differences were observed, the game may improve the consciousness of helping others. Our developed game was not an online game but an offline game, and this fact suggests that games can improve the consciousness of helping others, that is, mutual aid. Previous studies have shown that online gaming can result in strong social ties, extending these ties to offline activities [32,33]. These previous studies focused on online gaming; however, as far as a serious disaster game is concerned, offline gaming may result in strong social ties and extend these to real activities.

Furthermore, the game contents require revision. The game should include a broader range of scenarios, such as facilitating evacuation and assisting individuals with disabilities and the elderly. Currently, the game involves only carrying an injured or fallen person and evacuating with valuables. To improve the comprehensiveness, additional scenarios should be incorporated into the game.

Instructing others during an evacuation may be an important factor; however, in our developed game, the quiz component was designed to select the items for direction, with only two options provided for directing individuals outside the house and each question being relatively simple. Therefore, participants may not engage deeply with the questions, thus becoming ineffective for improving their consciousness through appropriate instructions. Hence, revising the quiz content and options is necessary. Additionally, providing options to present more challenging scenarios is necessary, thereby improving participants’ ability to provide effective instructions and enhancing their overall engagement.

6. Conclusions

During disasters, “self-help”, “mutual help”, and “public support” are important factors for sustainable disaster education. To raise the awareness of mutual aid, a game was developed to simulate evacuation from a fire while rescuing injured and fallen people and removing valuables. Next, the effectiveness of the developed game was evaluated through experimentation. Prior to and after gameplay, participants completed a questionnaire assessing their attitudes toward mutual assistance. We compared the scores from the pre- and post-game questionnaires. The results indicated that the game did not significantly affect participants’ willingness to cooperate with strangers, provide accurate instructions, or engage in mutual assistance.

Although no statistical effect was observed, the score for “helping each other” increased after gameplay, and the free descriptions indicate that the participants found the game realistic and favorable. Therefore, it is necessary to improve the game content and increase the difficulty level of the questions to improve the effectiveness of helping each other. Additionally, while the game was evaluated after approximately 10 min of play, continuous gameplay may reveal more pronounced effects. Therefore, we believe that a change in mutual aid awareness may occur after several days of gameplay.

The effect of gameplay was insignificant; however, we believe that the game has the potential to improve self-help, especially mutual help skills, regarding sustained disaster education. In fact, we have developed a game for disaster education and demonstrated its effectiveness in improving the consciousness of disaster prevention [34]. Therefore, as previously mentioned, we are confident that the game we have developed will show the desired effects if continuous play is implemented. In addition, the game is currently a PC-based game; if ported to mobile devices such as smartphones and the Nintendo Switch, it is expected to enable anyone, anywhere to enjoy and sustain training, leading to the improvement of self-help and mutual aid skills.

This study contributes to the existing game-based learning literature by introducing the concept of mutual aid as a focus for educational game development. Unlike previous research that has largely concentrated on self-help or individual preparedness, this study targets cooperative behaviors within communities during disasters. By combining simulation and gamification, this study introduces a novel learning tool that promotes collective action in the face of disasters. Future research should explore more diverse scenarios and apply this model to different age groups to validate the long-term effectiveness of this approach.