1. Introduction
Climate change has led to an increase in annual average temperatures across the globe, with 2016 being the warmest year on record (followed by 2015, 2017, and 2018) [
1,
2]. State-of-the-science climate models indicate that under a variety of scenarios, this warming trend will continue in the coming decades. For example, according to the IPCC (Intergovernmental Panel on Climate Change) Fifth Assessment Report (AR5), the increase of global mean surface temperature is likely to be 2.6–4.8 under RCP8.5 (the highest CO
2 emission scenario) by the end of the 21st century (2081–2100) compared to 1986–2005, which will certainly increase the frequency and duration of extreme heat events [
3]. In Korea, the increase in annual average temperatures was estimated to be up to 5.3 °C and the annual extreme heat wave duration was projected to increase by 6.1 days in the late 21st century compared to the late 1990s according to the most severe greenhouse gas emission scenario [
4]. Climate change poses extensive direct and indirect threats to public health; one of the primary effects of climate change is the health burden of high temperature and heat waves on mortality and morbidity [
5].
Numerous studies have reported significant impacts of high temperature on mortality and hospitalizations in Korean populations [
6,
7,
8]. Additionally, the impact of heat waves on the increase in mortality has been examined under different heat wave definitions based on the application of heat indices and the frequency and duration of heat waves [
9,
10,
11]. Studies have shown that vulnerable groups for mortality due to short-term exposure to high temperature include people living in urban cities without adequate air conditioning, the elderly especially those living alone, children, people in poverty, and people with underlying health conditions that are exacerbated by heat [
12,
13,
14,
15,
16].
Many countries, including Korea, have been developing and applying policies for preventing health burdens from extreme weather and improving resilience of societies to heat-related public health damages. Evidence-based policy making is crucial for continuous successful performance of these policies, as well as the development of additional health-protective policies in other countries that are likely to experience the effects of extreme weather, especially in the context of climate change.
In response to the growing concern of impacts of heat waves on public health, in 2008 Korea launched national heat wave health plans [
17] that are activated by the heat wave alert system administered by the Korea Meteorological Administration. The heat wave health plans aim to warn the public and health authorities of the impacts of heat waves, and to mitigate the associated negative health consequences [
18]. The plans have been developed with growing cooperation among related ministries and municipalities [
17]. Heat wave alerts are issued by forecasts for regions where daily maximum temperature is expected to be 33 °C or above for 2 or more consecutive days based on the alert system [
19]. One common threshold temperature of 33 °C is used for every region.
When a heat wave alert is announced, the heat wave department of the local municipalities immediately receives a warning bulletin from the Korean Meteorological Administration. While any person could be impacted by a heat wave, some implemented heat wave preventive actions are applied to groups that are anticipated to be particularly vulnerable, including the elderly (age 65 and more), disabled people, people who live alone, outdoor workers, and homeless peoples. The heat wave health plans incorporate various measures to be undertaken by the local government, such as opening shade shelters at night and on weekends/holidays and dispatching health care providers to the target vulnerable subpopulations. The Korean Ministry of Employment and Labor operates educational programs on heat-related health risks for construction and shipbuilding sites during warm seasons. When an alert is announced, ministries instruct local officials to operate break times (2–5 pm) at work sites for the elderly (age 65+) and outdoor workers (e.g., farmers, manual laborers, and soldiers), and at schools for young students. The Korea Center for Disease Control and Prevention operates ‘surveillance systems for heat-related illnesses’ during the warm season to monitor morbidity due to heat-related illnesses across the country. Further, it is instructed to supply heat-related first aid items (e.g., ice pack, saline solution) to local ambulances. While implementation of preventive measures by local municipalities can vary among regions, the national heat wave health plans have continuously grown by encouraging municipalities to develop and operate their own plans optimized for local situations.
While the significance of the relationship between heat waves and mortality has been demonstrated and national-scale plans have been executed many countries and cities [
20], little is known about whether the implementation of heat wave alert system and concurrently executed heat wave health plans (here on, referred to jointly as ‘the heat wave warning system’) produces a positive effect on reducing the mortality associated with heat waves. Evaluations of the heat wave warning system will inform evidence-based interventions aimed at reducing heat-related health impacts. We aimed to evaluate the effectiveness of the heat wave warning system to prevent the impact of heat waves on mortality in Korean populations for 2009–2014, utilizing the accuracy of the heat wave alert system in detecting actual heat wave days and the discrepancy between the early heat wave alerts and the actually monitored temperature.
3. Results
Table 2 shows characteristics of the mortality, heat wave, weather, and air pollution level data. The DD analysis included 50,002 all-cause deaths, 9917 cardiovascular deaths, and 4127 respiratory deaths. The number of all-cause mortalities during the eligible days for the analysis (50,002 deaths) was 25.7% of the total all-cause mortality for all days in the study period (194,409 deaths). The percentage of eligible days for DD analysis of the study regions (12,604 days) among all days in the study period (50,027 days) was 25.2%. The detection rate of the heat wave alert system for true heat wave (2202 days) or non-heat wave days (7,997 days) was 80.9% for the eligible days. The percentage of false negative heat wave days and false positive days for the eligible days was 6.1% and 13%, respectively.
The percentages of false positive and negative heat wave days by year showed slightly increasing false positive heat wave days and decreasing false negative heat wave days since 2009 (
Supplementary Table S2), implying a growing conservative approach of the alert system for detecting heat wave days. While Daegu is known as the hottest city in Korea, Daegu also showed the highest percentage of false negative heat wave days (3.85%) among the study cities.
Figure 1 visualizes the concept of the computed average all-cause mortality (person/day) of each heat wave category for eligible days for analysis and the DD (∆) implying the effect of the heat wave warning system. The computed average all-cause mortality for this figure is not yet adjusted with population size of study region and baseline characteristics of weather and air pollution. The difference of mortality between true heat wave days (
E[
Y1(1)]) (4.072 deaths/day) and false positive heat wave days (
E[
Y0(1)]) (3.752 deaths/day) was smaller than the difference of mortality between false negative days (
E[
Y1(0)]) (4.435 deaths/day) and true non-heat wave days (
E[
Y0(0)]) (3.936 deaths/day), which indicates a reduction in mortality risk of heat waves attributable to the heat wave warning system.
The estimated effects of the heat wave warning system on the risk of mortality of heat waves by population characteristics with adjustment of population size, weather, and air pollution are shown in
Figure 2 and Supplementary
Table S1. Despite the heat wave warning system, there was an increase in total all-cause mortality (1.687 per 1,000,000 population per day, [95% CI: 1.118, 2.255]) across all age groups for the study period (
Figure 2A). Increases in mortality were found for all-cause mortality for the subpopulations of job status (
Figure 2B) and marital status (
Figure 2C). Evidence of effects of the heat wave warning system on reduction in mortality risk of heat waves was found for several subpopulations of age and education level: people aged 0–19 (−0.555 deaths/1,000,000 population per day, [95% CI: −0.993, −0.117]) (
Figure 2A), and people aged between 19–64 who had no education (−0.114 deaths/1,000,000 population per day, [95% CI; −0.227, −0.061]) (
Figure 2D). Based on the interaction test using the central estimates and their standard error with a significance level of 0.05 [
34], the reductions in mortality in these subpopulations were statistically different from the effects of the heat wave warning system on mortality in the other age groups in each category (data not shown).
To describe the sensitivity to weight truncation, the sensitivity analyses using propensity score weights truncated at the 99th percentile were shown in
Supplementary Materials (Supplementary Table S3). Unlike the main result, under a less conservative setting with the truncation temperature, several additional subgroups showed significant reductions in all-cause mortality (indicating beneficial mortality effects of the heat wave warning system), including people aged 75+ who were unemployed and people aged 75+ who were widowed.
The estimated effects of the heat wave warning system on cardiovascular and respiratory mortality by age, sex, job status, marital status, and education are shown in
Table 3. Significant effects of reduced cardiovascular mortality from the heat wave warning system were found for people aged 75+ who were unemployed (−5.797 deaths/1,000,000 population per day [95% CI: −10.856, −0.739]), people aged 75+ who were widowed (−4.524 deaths/1,000,000 population per day [95% CI: −8.617, −0.431]), and people aged 19–64 with no education (−0.091 deaths/1,000,000 population per day [95% CI: −0.140, −0.042]). People aged 65+ who had university or higher degrees (−0.612 deaths per 1,000,000 population [95% CI: −1.140, −0.084]) and people aged 75+ who had education of university or higher degrees (−1.688 deaths/1,000,000 population [95% CI: −3.050, −0.325]) experienced a protective effect of the warning system. The reductions were statistically different from the estimates of the changes in mortality in the other age groups in each category according to the interaction test. Significant effects of reduced respiratory mortality were found for people aged 0-19 (−0.090 deaths/1,000,000 population per day [95% CI: −0.128, −0.053]), people aged 65+ who were widowed (−0.310 deaths/1,000,000 population per day [95% CI: −0.526, −0.094]), people aged 65+ who were single (−0.349 deaths/1,000,000 population per day [95% CI: −0.534, −0.163]), people aged between 19–64 who had no education (−0.032 deaths/1,000,000 population per day [95% CI: −0.055, −0.009]), people aged 65+ who had elementary education (−0.884 deaths/1,000,000 population per day [95% CI: −1.582, −0.185]), and people aged 75+ who had education of university or more degrees (−0.999 deaths/1,000,000 population per day [95% CI: −1.947, −0.050]). Except for the estimates for education attainment, differences in the estimated effects of the heat wave warning system among age-subgroups were significant for these reductions in respiratory mortality.
To illustrate potential changes over time, the DD analysis was applied to the two 3-year periods (2009–2011, 2012–2014) for the subgroups that showed significant effectiveness of the heat wave alerts on reducing mortality throughout the study period. The results (
Table 4) showed decreases in mortality after the implementation that were attenuated to null over time for some groups such as people with elementary education (aged 65+ or 75+) and people who were widowed (age 65+ or 75+). On the other hand, people aged 19–64 with no education experienced delayed effects of the heat wave warning system on reducing mortality in the later period (2012–2014). Cardiovascular mortality for those who were widowed (aged 75+) also showed a delayed effect of the system.
The sensitivity analysis applied to finer age subgroups in children found that the significant effects of the heat wave warning system on reduced all-cause or respiratory morality for children aged 0–19 were mainly due to the reduced mortality for children under age 5 (
Supplementary Table S4). Due to small number of deaths, the DD estimates were not estimated for respiratory mortality in children aged 5–19.
4. Discussion
Many countries have implemented nationwide or state- and local-level heat wave warning systems to minimize the risk of heat-related health problems [
35]. While numerous epidemiologic studies focused on examining the relationship between high ambient temperature and mortality, few studies have reported changes in mortality risk during serial heat waves over years as a result of the increased awareness of the risk of heat in relation to the implementation of a heat wave warning system [
36,
37,
38,
39]. A study of Quebec, Canada applied a quasi-experimental method to evaluate the effectiveness of the Canadian heat action plans on mortality [
40], finding reduced mortality attributable to the plans for elderly people and people living in low-education neighborhoods. A recent study examined if days with national heat wave alerts announced had significantly lower mortality risk compared to the comparable days but without alerts in 20 US cities using a case-crossover approach and found no significant association between the heat alerts and reduction in mortality [
41]. To the best of our knowledge, our study is the first to assess the effectiveness of the national heat wave warning system and the subsequent governmental heat health plans on reducing mortality risk during heat waves by sub-groups of biological and socio-economic characteristics using a DD model combined with propensity score weighting approaches.
We found that the percentages of false negative and positive heat wave days (i.e., days with inaccurate heat wave alerts) varied by cities and years. Particularly, higher percentages of false negative and false positive heat wave days in Daegu, known as the hottest urban city in Korea, cast a substantial public health implication of heat-related health outcomes. In addition to the different exposure-response relationships for ambient temperature and mortality among regions, the variation of accuracy of the heat wave alerts may imply the need for regional-specific heat wave criteria.
Our results did not find a beneficial effect of the heat wave warning system for mortality of all-ages group in major 7 urban cities. These results may imply an incomplete achievement of public health benefits of the heat wave warnings and health plans, and suggest the opportunity to adjust the plans to maximize improvements to public health. However, we identified benefits for all-cause mortality in some subgroups. The negative DD estimates for all-cause mortality were coherently found in older persons (age 75+), which implies potential effectiveness of the heat wave warning system for reduced heat-related mortality for the target older age groups with vulnerable conditions (e.g., unemployment, being widowed), although the statistical power and the magnitude of the results were small. The additional significant estimates of DD for older subgroups under less conservative settings (i.e., by the 99th percentile truncation of the propensity scores) emphasize that our main results obtained in strictly conservative settings still claim significant decreases in mortality especially in older population. The positive DD estimates may indicate that mortality occurred beyond the level at which the intervention could function as a buffer preventing heat-related mortality. In the perspective of public health, the DD estimates for all-cause were constantly positive for younger old age group (age 65+) and young adults (aged under 65), which indicate that mortality risk from heat waves still remain for these subgroups in spite of implementation of the heat wave warning system.
The weak statistical power of the results for all-cause mortality in the elderly may be related with the relatively large fraction of deaths from tumor/neoplasm. According to the national statistics 2017 [
42], the major cause of death in people aged 65 and above was tumor/neoplasm (26%) followed by cardiovascular diseases (24%), respiratory diseases (14%), and senility (6%). Our study suggested significant results for beneficial effects of the intervention component of the warning system on cardiovascular and respiratory mortality risk of heat waves in older persons (age 75+), particularly for those with additional susceptible factors such as unemployment, living alone, and low education attainment. In the context of vulnerability, living alone, having a lack of mobility and air-cooling system, and living on the top floor are considered to increase vulnerability to heat waves [
43]. These factors are often correlated with age, which is also one of the main effect modifiers associated with vulnerability to heat-related mortality [
44], while the evidence of sex as an effect modifier is less consistent [
33,
45]. The majority of heat wave health plans in many countries are aimed at the population over the age of 65 or 75 years and implement preventive measures for mitigating the health burdens within this vulnerable groups [
46]. Targeting vulnerable groups including elderly population gives rise to a question on the difference in the changes in health impacts of heat waves across various age groups [
44]. Our results indicate that elderly populations may have benefited from the heat wave health plans targeting this subpopulation. Also, our results support the previous study [
47] of Korea that proposed a temporal decrease in heat-related mortality risk in elderly population (age 75+) after 2008 in which the national heat wave warning system was launched. As the age increases, prevalence of chronic diseases and activity limitations increase, resulting in differences in the health status between relatively younger and older elderly people. Given the rapid aging phenomenon in Korea in which more accessible health care for the elderly is demanded, our results showing the effects of heat wave warning system in the older elderly group (age 75+) are particularly interesting.
Furthermore, our results found that there was a temporal trend in the effectiveness of the heat wave warning system for those subgroups that experienced reduced mortality attributable to the heat wave alerts. A possible explanation for the trend is that people who immediately benefited from the heat wave warning system after its implementation were desensitized to the alerts and preventive measures. For the delayed effectiveness of the heat wave warning system for several groups, it is possible that a certain latent period existed for the warning system to be in effect. Given the ongoing reinforcement of the heat wave warning system, continuous examination in temporal trends of the effectiveness of the warning system intervention is crucial.
Due to differences in regional administrative resources and discretion for recommended governmental actions during heat wave days, the performance of the governmental heat wave health plans during heat wave days can vary by administrative regions [
41]. Evaluation of regional performance of heat wave health plans is methodologically challenging. Although governmental officers may be concerned about the actual adherence of local offices to the national heat health plans, we did not have information on the performance of local offices for the national heat health plans and the presence of local specialized preventive actions for heat waves was not available. Future work could examine these issues to differentiate how specific implementations of heat wave warning systems affect health impacts.
Behavioral changes after heat wave warnings may explain our findings of significant decreases in all-cause mortality in children aged 0–19 and young adults aged 19–64 with no education. An early access to prevention information such as leaflets may contribute to substantial individual behavioral changes, such as increased use of cooling systems and wet clothes in elderly people [
48]. Voluntary actions performed by individuals who had access to the heat wave announcement from the media and were aware of preventive actions to reduce the health effects of heat waves could contribute to the reduced mortality during heat waves. Such actions include staying inside or under shade and drinking a sufficient amount of water. However, protective behaviors may not necessarily be due to heat wave announcements [
20]; those who do not view themselves as vulnerable to heat may not take protective actions irrespective of actual vulnerability. In some cases, even people who consider themselves as vulnerable to heat may not change their behavior [
41,
49]. On the other hand, those who take voluntary preventive actions may do so not because of the heat wave alerts but due to their perception of hot weather [
20]. Our analysis addresses this issue by distinguishing between heat wave days that are identified as those with heat wave warnings and those without warnings. Evidence on the awareness and tendency of taking preventive actions in population is not yet available in Korea; such information would allow more a stable investigation of the association between individual-level behaviors and improved health during heat waves, and thereby further inform which behaviors are most effective.
In our study, significant evidence of the effectiveness of the heat wave warning system for reducing all-cause mortality related to heat waves was not found in white-collar nor blue-collar workers. Significant reduction in respiratory mortality was found in older blue-collar workers (i.e., aged 65+ or 75+). Numerous studies have reported significant impacts of heat waves or high temperature on mortality among workers such as farmers and labor workers at construction/extraction sites whose job is more likely related with performances at outdoors or hot environments [
50,
51]. Some previous work suggested that younger workers are more vulnerable to occupational injuries because they are likely to undertake more strenuous tasks, lack key training, and have not yet developed the same skills [
52,
53]. The prevention of heat-related illness and mortality at work sites can be improved by employer’s active support for occupational health preventive measures in addition to public health plans targeting the public. Preventive measures that are practical at work sites include monitoring the temperature and thermal stress indices, cooling systems, water intake management, shaded rest areas, and appropriate rest cycles and clothes [
51,
54]. The Ministry of Employment and Labor of Korea has announced the worker health prevention plans for heat waves, including the preventive measures mentioned above, and recommended that employers follow the plans since 2009 [
55]. Further impact evaluation studies for the occupational health risks of heat stress would help develop evidence-based intervention programs targeting highly vulnerable occupations.
Our results showed a significant reduction in cardiovascular and respiratory mortality attributable to the heat wave warning system in people aged 75+ years who were widowed. The effect of marital status was less robust to the selection of cut-offs for truncation of propensity score weights, likely due to small sample size. Generally, a married people may be less vulnerable as they could be cared for by their partners during hot periods. Evidence of the effect of marital status on the mortality risk of heat waves is relatively limited. Previous studies suggested an increased mortality risk in persons who are married or widowed/divorced during heat waves in Korea; São Paulo, Brazil; and northern China [
56,
57,
58], whereas a study in Michigan showed higher risks of cardiovascular mortality during hot days in the unmarried elderly population [
59]. A study in King County, Washington, US, did not find that marital status altered the mortality risk on hot days [
60].
The main analysis results rely on the parallel trend assumption of the DD estimates. However, the changes in mortality during non-heat wave days may not be equal to the mortality change during heat wave days. The DD estimate can be positive even though there is in fact some reduction in heat-related mortality. A situation where the reduction in mortality during non-heat wave days is solely attractable to reduction in outdoor activities of the target population, which is an implausible assumption in reality but made for the discussion. In such a situation, we could consider a group of people who avoid outdoor activity during heat wave days regardless of the heat wave warning, but avoid it during non-heat wave days only when there is the warning. This group would have no effect of the warning during heat wave days, but some effect during non-heat wave days, which is a violation of the parallel trend assumption. For this subpopulation, since they will not participate in outdoor activities anyways during heat wave days, the warning system is not effective during heat wave days, but is effective during non-heat wave days. This violation can be severe if the proportion of this subpopulation is large but information on this proportion cannot be observed. Further research is required to estimate this proportion to check the plausibility of the parallel trend assumption. Further quantifications of the effectiveness of the intervention during the non-heat wave days with and without heat wave alerts (i.e., the counterfactual itself in our study setting) will be helpful to examine if the heat wave warning system plays a role as a preventive measure for days that are not extremely hot throughout warm seasons and contribute to reduction in mortality.
Although the statistically significant negative DD estimates coherently found in the elderly support the evidence of the beneficial mortality effects of the heat wave warning system for the elderly group, our results on the benefits of the heat wave warning system may necessitate cautious interpretation. The multiple testing problem (increases in the probability of wrongly rejecting a null hypothesis when performing multiple simultaneous hypothesis tests) [
61] may have emerged as the DD method was applied to multiple different subpopulations. Nonetheless, the multiple testing problem was not prioritized over examining subgroup effects in our study, as our research aimed to examine how the effects of the heat wave warning system vary with the levels of vulnerability factors such as age or occupation. Multiple testing correction methods such as Bonferroni or Holms corrections give fewer insights on variation in vulnerability of subgroups. While the DD estimate is the specific linear combination of four quantities and the variance of it is the sum of each quantity’s variance, applying multiple testing corrections would result in very wide confidence intervals. Moreover, the results adjusted by multiple testing corrections would not provide informative insights on subgroup vulnerability as the DD estimates in our main results were relatively smaller compared to their large variance. We consider that our propensity score weighting method can provide reliable variance estimation while diminishing the potential confounding and selection biases for the effect estimates. With this context, our various subgroup analysis should not be considered as hypothesis generating research and finding hypotheses is not the main goal of this study. Future work examining the benefits of the heat wave warning system based on larger sample size and sophisticated statistical methods such as the bootstrap method would be useful to further verify the findings of this study.
This study has several strengths. By combining temperature monitoring data and heat wave announcement data, we could determine the effectiveness of the national heat wave warning system for preventing mortality during heat waves. Our novel methods using the DD method combined with propensity score weighting enabled assessment of the changes in mortality that were attributable to the national heat wave warning system, with control for measured (e.g., air pollution) and unmeasured confounders. This provides convincing evidence to inform policy decisions [
25]. Also, our methods, which are reasonable to apply to a warning system that is imperfect for detecting the days needing warnings, provide opportunities for evaluating heat-related public interventions, while only a few studies with limited geographical locations and populations have evaluated changes in heat-related health outcomes attributable to an administrative intervention using a quasi-experimental research design [
40,
62]. Another quasi-experimental design that can be feasible for the effect of heat wave warning system is the regression discontinuity model that can be used for intervention based on a continuous measure with a clearly defined threshold value for eligibility of enrollment to the intervention and the effectiveness of interventions in the vicinity of the threshold [
25]. We note that this method was not applied to our study as our study question is if and how the heat wave warning system affects the mortality risk due to heat waves during the entire summer season in the whole population. Also, we reduced selection bias from incomparable comparison days to the hot enough days for heat wave warning announcement, by limiting the analysis to the identified eligible comparison days based on measured covariates. Another strength of this study is the investigation of impacts for subgroups of socio-economic status.
Our study has some limitations. Our study is based on heat wave warnings, but does not address potential changes in individual behaviors that may occur in conjunction with heat wave warnings. We also did not have information of the ways in which the heat wave warnings were implemented in each region, which may have differences and reduce the comparability of changes between cities. We assumed that the monitoring stations within a given city represent the most accurate weather condition of the Gu, a smaller administrative unit than City. The same exposure level of all individuals within a city was assumed, using available monitoring data and heat wave announcement data; however, actual intra-city heat exposures may differ in an urban environment, especially due to the urban heat island effect. We focused on urban regions in an effort to have higher similarity in population for an appropriate study design; future work could provide comparisons of heat wave effects in urban areas versus rural areas. Lastly, we assumed that the heat wave alerts and subsequent governmental actions are activated according to the latest announcement of the heat wave alerts, for which the integrity (or level) of heat wave can be updated. Although this assumption was reasonable considering the update of information at the website of the Korea Meteorological Administration, it is not certain how urgently local officials receive the order for implementing heat wave health plans. Research that links heat wave warnings and implementation measures to health is needed to confirm these findings.
In summary, our study provides information on whether the heat wave warning system has contributed to a reduction in cause-specific mortality and which subpopulations benefited more from the heat wave warning system in Korea. The results indicated that several vulnerable subgroups experienced reduced all-cause mortality. The reduction in mortality might have been achieved through the heat actions plans implemented by local administrative officials and behavioral changes of individuals during days with an announced heat wave alert. Future study efforts are imperative to disentangle the effectiveness of these multiple pathways and find the most effective preventive measures for regional climate and local administrative capability. Continuous efforts on exploring changes in the effectiveness of the heat wave warning system over time are required to further support evidence-based policy making in relation to the health threats of climate change.