Disaster Response System Dynamic Assessment Model Based on Questionnaire Investigation: A Case Study of Nuclear Accident Preparedness Education ^†

Abstract

Based on the results of the nuclear accident response education questionnaire, we use system dynamics to develop a nuclear accident response education effectiveness evaluation model. A questionnaire survey was conducted for the pre- and post-tests of the course to understand course effectiveness and explore the interactive relationship between factors. The system dynamics is used to develop a nuclear accident response education effectiveness evaluation model. Age and education are significantly related to education and training experience (number of times), and the age, occupation, and education level are significantly related to individuals’ experience (number of times) of participating in a nuclear accident evacuation exercise. In the workshop courses, the public’s awareness of self-protection against nuclear accidents was significantly improved. Experience in educational training was significantly related to evacuation exercises. Individuals’ age and experience of participating in education and training can be used to predict their willingness to participate in evacuation exercises. Using systems thinking and analysis, an evaluation model for nuclear incident response education effectiveness is constructed as a reference for evaluating effectiveness. The designed education and training courses can increase public participation in nuclear accident response education and strengthen the cognitive benefits of response protection.

1. Introduction

According to the International Atomic Energy Agency (IAEA) “2022 Annual Report”, there are 438 nuclear power plants in operation in 32 regions around the world [1]. The only operating nuclear power plant in Taiwan is in the decommissioning stage and will cease operation in 2025. However, the decommissioning process in nuclear power plants takes a long time, and it is necessary to strengthen people’s concept of and their response to nuclear accidents continuously. The concept of disaster prevention in education is based on reducing the risk of disaster, improving preparedness, and strengthening recovery rapidly after the disaster.

Education is important as it can change people’s conceptions and attitudes about the risks of nuclear power. People who are opposed to nuclear power misunderstand nuclear power [2]. Tsai et al. [3] pointed out that the application of gamified courses in higher education has received widespread attention. Compared with regular school courses, experiential learning helps participants reduce their feelings of rejection and boredom. The use of the game mode improves the learners’ understanding of nuclear power [4]. Han et al. [5] also proposed that the attitudes of students towards nuclear power can be changed through education.

When a nuclear accident requires the evacuation of people, their degree of cooperation affects the smoothness of the evacuation operation. Seddighi et al. [6] mentioned that the capabilities developed by education can reduce the risk of disaster. Shaw et al. [7] believe that disaster prevention education includes the education of students in schools and the education of adults in the community. Ismail-Zadeh et al. [8] believe that one of the ways to reduce the risk of disaster is to convey knowledge about disaster response and its impact, such as disaster reduction and preparedness, to decision-makers and those directly exposed to risks.

Using interactive teaching materials, we develop a method to instruct public courses and survey the government’s public protection plans for nuclear accidents before and after the course to evaluate the effectiveness of interactive nuclear safety education.

2. Materials and Methods

2.1. Questionnaire Design

A nuclear safety education and training questionnaire was created for the pre- and post-test surveys based on the course content and the public’s understanding of it (Table 1).

Table 1. Nuclear safety education and training course content and teaching.

Item	Course Content	Comprehension Level of Nuclear Safety Education and Training Courses
1	Knowing the nuclear accident public warning system	Do you know the nuclear accident siren?
2	Planning the nuclear accident assemble location and functions	Do you know the public assembly location of nuclear accidents?
3	Nuclear accident evacuation bus and route planning	Do you know how to take the nuclear accident evacuation buses?
4	The plan for nuclear accident’s public evacuation shelters	Do you know the plan of nuclear accident evacuation shelters for people?
5	Who and when to take iodine tablets in a nuclear accident	Do you know the standard for taking iodine tablets during a nuclear accident?

Those who participated in nuclear safety education and training in the emergency planning zone of the third nuclear power plant participated in this study. In total, 296 questionnaires were recovered, with 291 being valid questionnaires, resulting in a recovery rate of 98.3%.

2.2. Data Processing and Analysis

We used a t-test to compare the pre-and post-test results of the education and training courses. A paired samples t-test was conducted to compare the mean difference between two groups of dependent samples.

We collected the questionnaire results and conducted data processing and analysis using SPSS 22. The analysis methods included descriptive statistics, the chi-square test, correlation analysis, a paired samples t-test, and regression analysis. The significance level of each statistical test was set at α = 0.05.

2.3. System Dynamics Assessment Model

System dynamics is based on system thinking. It is mainly used to understand and analyze dynamic systems composed of multiple interactions. It is a process-oriented research method. The system dynamics assessment model simulates and predicts the system behavior by building models. The construction of the model relies on Causal Loop Diagrams (CLDs) and Stock Flow Diagrams (SFDs). Causal loop diagrams are used to describe the interactions between variables in a system and their feedback loops. Feedback loops can be positive or negative, enhancing system behavior or weakening system behavior.

3. Results and Discussion

The demographic variables and past participation experience were analyzed using a chi-square test, bivariate correlation analysis, a nuclear safety education and training course effectiveness t-test, and stepwise multiple regression analysis.

3.1. Demographic Variables and Participation Experience Analysis

Of the 291 participants, 66.3% were females, 86.2% were middle-aged and elderly, 43.6% were homeowners, 20.6% worked as freelancers, and 57.7% had the education level of junior high school or below.

In total, 200 participants (68.6%) had experienced nuclear safety education and training; 46.4% had participated in nuclear safety evacuation exercises; and 53.6% had partaken in neither.

We conducted a chi-square test on past participation experience to verify whether different demographic variables are related to their participation behavior. Age (x²(12) = 28.432, p = 0.005 < 0.05) and education level (x²(6) = 12.965, p = 0.044 < 0.05) were correlated with “nuclear safety education and training”.

Age (x²(12) = 28.336, p = 0.004 < 0.05), occupation (x²(15) = 25.784, p = 0.040), and education level (x²(6) = 14.794, p = 0.0022) were correlated with the participants’ experience participating in “Nuclear Safety Evacuation Exercise” (Table 2).

Table 2. Cross-contingency analysis table of past participation experience among nuclear safety education and training participants.

Variables	Nuclear Safety Education and Training Experience (Number of Times)	Variables	Nuclear Safety Evacuation Exercise Experience (Number of Times)
	χ2		χ2
Gender	1.555	Gender	7.120
Age	28.432 **	Age	28.336 **
occupation	13.853	occupation	25.784 *
Education level	14.794 *	Education level	14.794 *
Gender	1.555	Gender	7.120

* as p < 0.05, ** as p < 0.01.

Based on the participants’ experience of undertaking nuclear safety education and training courses, the correlation between variables showed a significant correlation (r = 0.683, p = 0.000 < 0.001) between the participant’s experience of undertaking nuclear safety education and training and nuclear safety evacuation exercises.

3.2. Bivariate Analysis of Training Benefits

The scores of the participants’ questions before and after the nuclear safety education training were evaluated using the t-test. The averages of the two samples were 7.964 and 9.233, and the correlation coefficient between the samples was 0.406 (p = 0.000 < 0.001, reaching significance). In the paired analysis of the pre-test and post-test, t(290) = −18.760, p = 0.000, the test result was found to be significant. The results of the paired sample analysis before and after the nuclear safety education training are shown in Table 3.

Table 3. Paired sample analysis before and after nuclear safety education training.

Mean	Standard Deviation	Standard Error	95% CI	t-Value	Degree of Freedom	Significance
−1.540	1.400	0.082	[−1.701, −1.378]	−18.760	290	0.000 ***

*** as p < 0.001.

3.3. Prediction and Analysis of Nuclear Safety Evacuation Exercise

Stepwise regression analysis was used to load the predictor variables, check the changes in the explained regression coefficient R² of the predictor variables, and select the most appropriate model. Multi-collinearity was tested, and the tolerance and variation inflation factor (VIF) were used for observation. The tolerance was between 0.721 and 0.964, and the variation inflation factor was between 1.037 and 1.388. The tolerances of this model were higher than 0.2, and the VIF was less than 4, showing multivariate collinearity. The Durbin–Watson test value was 1.733, indicating that the residuals were independent with no autocorrelation.

Model 1 showed the prediction and analysis results of the demographic variables with regard to participation in nuclear safety evacuation exercises. The prediction variables were gender, age, occupation, and education level. Before performing regression analysis, the population background was normalized to meet the requirements of the regression analysis. The occupation category was divided into two categories: “Housekeeper or Retired” and “Employed”. Age (β = 0.225, t = 3.722, p = 0.000 < 0.001) explained the variation in the participants’ participation in nuclear safety evacuation exercises (R² = 0.059, F(4,286) = 4.502, p = 0.000 < 0.001).

Model 2 used past nuclear safety education and training experience as predictor variables. Age (β = 0.099, t = 2.179, p = 0.030 < 0.05) and past nuclear safety education experience (β = 0.667, t = 15.436, p = 0.000 < 0.001) explained the variation in the participants’ participation in nuclear safety evacuation exercises (R² = 0.488, f(5,285) = 54.242, p = 0.000 < 0.001). The explanatory power of Model 2 was 48.8%. The analysis results are shown in Table 4.

Table 4. Regression analysis of demographic variables, nuclear safety education and training on nuclear safety evacuation exercise.

	Model 1			Model 2
	B	SE B	β	B	SE B	β
Constant	1.045	0.274		0.189	0.210
Gender	−0.091	0.122	−0.048	0.006	0.091	0.003
Age	0.185	0.050	0.225 ***	0.081	0.037	0.099 *
Occupation	0.068	0.122	0.037	−0.037	0.091	−0.020
Education level	−0.072	0.076	−0.057	−0.097	0.056	−0.077
Training experience				0.645	0.042	0.667 ***
R2	0.059	0.488	0.059	0.488	0.059	0.488
Adj R2	0.046	0.479	0.046	0.479	0.046	0.479
F for change in R2	4.502 **	54.242 ***	4.502 **	54.242 ***	4.502 **	54.242 ***

* as p < 0.05, ** as p < 0.01, *** as p < 0.001.

3.4. System Dynamics Evaluation

A system dynamics model was used to develop an evaluation model of nuclear safety education effectiveness based on the five major themes of the public nuclear safety education content: public warning system, assemble locations, evacuation methods, evacuation planning, and the time at which iodine tablets should be taken. The factors related to public nuclear safety education were used to construct an institutional structure model for evaluating the effectiveness of nuclear safety education (Figure 1).

Figure 1. Model for effectiveness assessment of nuclear safety education using systems thinking.

3.5. Discussion

The chi-square test was used to verify the correlation between the demographic variables and past participation in nuclear safety education and nuclear safety evacuation exercises. Age and education level were significantly related to the number of times the participants had participated in nuclear safety education and training. Age, occupation, and education level were significantly related to the number of times the participants’ had participated in nuclear safety evacuation exercises.

Most of the participants in nuclear safety education and training were middle-aged/elderly people (over 51 years old), with education levels below junior high school (inclusive). Those who had participated in nuclear safety evacuation exercises were middle-aged/elderly people (over 51 years old) who were professionals or who had family members who were professionals. Most of them were managers, retired, or self-employed, and their education level was junior high school or below. Those who participated in nuclear safety education courses and nuclear safety evacuation exercises had less need to consider their employment status, so they were more willing to participate in the courses or exercises.

The post-test scores were higher than the pre-test scores, indicating that nuclear safety education and training courses led to a significant difference in people’s understanding of self-protection against nuclear accidents. There was a significant correlation between the participants’ experience participating in nuclear safety education and training and their experience participating in nuclear safety evacuation exercises (r = 0.683, p = 0.000 < 0.001, moderate correlation).

Multiple stepwise regression analysis was used to explore whether the demographic variables and the participants’ past experience of nuclear safety education had an impact on their participation in nuclear safety evacuation exercises. Age (β = 0.099), past participation (β = 0.667), and the standardized regression coefficients were positive, showing a positive relationship. The older they were and the more experience they had, the more experience (willingness) they had in participating in nuclear safety evacuation exercises.

To understand the factors that affect the effectiveness of nuclear safety education and their correlations, an analysis framework was constructed to evaluate its effectiveness in improving disaster prevention for the public in the future.

4. Conclusions

Nuclear accidents are unfamiliar to the public in disaster prevention promotion education courses. Since the launch of the public protection course for nuclear accidents, the public’s understanding of nuclear accident preparedness and response measures has gradually improved. However, when promoting courses about nuclear accident public protection in the community, demographic statistics must be considered. Therefore, the course design and production of teaching materials for disaster prevention education must be “life-oriented” and “interesting”. A “content localization” curriculum needs to be offered to improve the goal of nuclear accident prevention education.