A Multi-Risk Approach to Climate Change Adaptation, Based on an Analysis of South Korean Newspaper Articles

: The risks caused by climate change are worsening worldwide, and it is recognized that national and regional responses to climate change are essential. This study therefore explores climate change risks that have been recognized as fatal to people and the environment by analyzing multi-inﬂuence factors that appear in multiple risk indicators. The climate change risks in this study are based on 73 existing risk indicators; the frame data for multi-inﬂuence risk factors are based on 3098 newspaper articles published over 24 years on the impact of climate change in South Korea. The main outcomes for this study were ﬁnding climate change risk trend from newspaper articles regarding climate change impacts through text-mining, and ﬁguring out the multi-risk indicators that are likely to occur at the same time with other risk indicators using network analysis. From the network analysis, we found that the major risk indicators have a high degree of interrelationship among risk indicators, including “increase in mortality rate from disaster”, “increase in ﬂood areas due to coastal ﬂooding”, and “destruction of repair facilities due to ﬂooding (river bank, etc.)”. The main risk indicators derived from this study can therefore be used as a reasonable standard when identifying the main risks posed by climate change and deﬁning future adaptation planning priorities.


Background
As scientific evidence on climate change accumulates and actual damage increases, awareness of climate change is spreading worldwide [1][2][3][4][5]. The international response strategy for climate change is being reorganized into a risk management system that predicts and manages uncertain future losses [6,7]. Climate change has continued to dominate both the political and business agendas for many years [1]. In the 21st century, climate change has been dubbed one of the biggest market failures the world has ever seen [8].
Risk management is a practical way to cope in a timely manner with extreme events [9]. Within the context of climate change, the risk management of extreme events is considered an interdisciplinary problem, and there has been discussion of its various aspects [10]. The growing literature dealing with climate change risk frameworks and risk identification aims to define exactly what climate change risk means in advance and to cope with risks [11][12][13][14][15][16]. According to [6], these climate change risks are dependent on exposure and vulnerability; certain climate change impacts are based on the concept that a risk consists of hazard, vulnerability, and exposure. Therefore, climate change risk management is intended to reduce the probability of incidents and trends and to reduce risk consequences by Table 1. Concepts of risk drawn from previous studies.

Researchers
Climate Change Risk Concept Characteristics [30] Consists of three components: "outcome", "likelihood", and "severity" Adds "seriousness" to existing risk concepts [31] Increases the likelihood of climate-related damage and loss Applies opportunity terms to risks (positive effects of climate change) [12] Expressed as a function of climate hazard and vulnerability - [32] An unprecedented challenge to human adaptability - [33] Expressed in terms of vulnerability and exposure to extreme climate change - [25] A potential negative impact or outcome that may occur in the present or future Applied time concept (present and future) [23] The realization of scientific techniques and empirical means, among many ways to face an uncertain future; a way of expressing predictions of the future -As these climate-change risks become more severe, questions about how to better assess, communicate, and respond to climate change risks at the community level have emerged as key issues within climate risk management [34]. In this context, a key step in risk management is risk assessment-the process of analyzing and assessing risk and opportunity elements based on identified or perceived risks. Prior research [27,35] on climate risk emphasizes the importance of risk assessment in climate change adaptation planning. While risk is not essentially a quantitative concept [25], the premise of assessing risk implies that risk-specific quantification is possible through quantitative analysis. Typically, the UK has established a risk matrix to minimize the impact of climate change and attempted quantification using risk indicators through Climate Change Risk Assessment (CCRA) [14]. CCRA specifies the risk management process and method for strategically responding to climate change, taking uncertainties into consideration in its assessment. This report compares and prioritizes approximately 700 climate change risk indicators to develop its climate change adaptation policy. The approach identifies various climate change risks, and can help policy makers reach reasonable conclusions when developing and prioritizing policies. References [14,36] have also developed climate change risk indicators, which illustrates various types of damage that may occur in the future, and compares the degree of risk. Since climate change studies involve primary and secondary risks that may arise in various fields and regions, further research is needed. Risks are thus invested with an array of meanings and levels of significance. To define exactly what is at stake and how it should be dealt with presents a rather complex, mutable, and contested problem within and across communities [34]. One thing that has been overlooked in the discussion of climate change risk issues is the fact that primary risks can be changed or extended into secondary risks and multi-risks that relate to economic, ecological, social, or other areas [37]. Some studies [37][38][39] have emphasized multi-risk interactions, but this field lacks relevant research, due to the uncertainty of climate change and the difficulty of quantifying multi-risk areas. According to the [6] climate change risks can be multiple, depending on the climate hazard or socio-ecological system; for this reason, scientific research and systematic responses and management are needed. However, due to the differing characteristics of hazards, few quantitative models exist that suit a fully multi-risk perspective [38]. Therefore, this study is derived from two main arguments. One is the argument that every climate change risk indicator should be designed for risk assessment and management, and the other is that a multi-risk perspective must be considered, as many existing studies [37,38,40].

Media Coverage of Climate Change Issues
Studies of climate change have increasing used newspaper articles [3][4][5][41][42][43][44][45][46][47]. The news media have been, and will remain, important in communicating information to the public and decision makers about the science of climate change and solution strategies [1]. According to [48], the public is more likely to notice weather extremes than long-term incremental changes in mean climate variables. Newspapers have been criticized for not being able to grasp objective information, and for reflecting political tendencies, biases, context, and values as well as reported data [41,47,49]. However, it is important to use newspaper articles in studies dealing with climate change because they provide information on the impact of climate change in various fields across the whole of society at once. The authors of [2] have investigated 14 years of newspaper reporting of climate change in Russia to suggest implications for climate communication. One notable result of that study is the finding that national economic conditions have a major impact on climate change reports. Further extending the geographical range, [47] have investigated media coverage of climate change in 27 countries, comparing the number of articles to each country's climate policy. This study has shown that media attention is especially high in carbon-dependent countries with commitment, an indication that media coverage is highly dependent on national politics and policies. While climate imagery proliferates, media analyses of climate to date have focused almost exclusively on textual representations. Reference [47] analyzed visual images attached to online articles about climate change between 2010 and 2013 in American, British, and Australian newspapers to identify broad patterns in the newspaper coverage of climate change. Reference [48] used Twitter to analyze how the mass media responds to "time variables", the "number of climate change publications in the mass media", and "temperature variables". Of these, the most important variables were found to be the temperature variables, suggesting that media coverage partly controls public attention. There is a rising scientific consensus on climate change, although climate change skepticism is constantly in the news [50,51]. Reference [52] dealt with skepticism using Twitter. In particular, they have analyzed the stream of Twitter conversations about climate change over two years, focusing on "hoax frames" that question the reality of climate change. Like these studies, media-based climate change research has mainly framed political concerns and energy issues, depending on content analysis. Although it is possible to extract varied in-depth information on climate change risks from newspaper articles, few studies have dealt with climate change risks. Future climate change studies should use techniques such as text mining to analyze newspaper articles, deriving high-quality information to predict and minimize risks [18,22]. Text-mining, a big-data research method, has been used in a growing number of studies [42,[53][54][55] to acquire and structure detailed information. Reference [53] has said that text mining allows researchers to extract information and trends from large amounts of textual data. Studies using text-mining techniques to explore climate change issues have recently been increasing [3,43,45,52]. The use of text mining in risk assessment, in particular, has been recognized as a superior way to identify high-frequency risks and to produce data that can be used to analyze the interrelationships between risks [18]. In this study, we have carried out a climate change risk analysis of 3098 newspapers using text mining. Furthermore, by analyzing each risk's complex relationships, we have compiled the risk indicators with a high correlation between risks. This can suggest the implications of climate change adaptation plans and help to anticipate and reduce climate change risk.

Research Hypotheses
The general risk assessment methodology is based on the following basic procedures, particularly in relation to quantitative analysis. Taken in order, the investigation covers hazard assessment, vulnerability assessment, and risk assessment [13,56]. These risk assessments have primarily been conducted as single-risk analyses, each targeting a specific area. Single-risk analysis can determine the individual risk arising from one particular hazard or the process occurring in a specific geographical area during a given period of time [37]. However, it is difficult to compare multiple risks in the same hierarchy or to evaluate multi-risks arising from various drivers or hazard. For this reason, single risk analyses are unsuited to comparing or selecting optimal climate change adaptation policies to deal with climate change risk. This study has therefore adopted a top-down approach, using existing risk indicators created by experts as basic data and applying them directly to policies. This approach facilitates the comparison of risk indicators, and suggests the applicability of a multi-risk perspective, unlike previous studies.
This study addresses the following three research hypotheses: 1.
Climate change risk indicators and actual climate change impact data (from newspaper articles over 24 years) have a significant correlation.

2.
Based on the concept of climate change risk, the degree of risk can be quantified using climate drivers such as heatwaves and droughts.

3.
Some risk indicators in this study will have multi-risk characteristics that occur simultaneously during a climate change event.

Climate Change Risk Indicators
Climate change risk indicators, a subject explored in this study, are derived from existing research [21]. In that paper, 81 climate change risk indicators for each sector were obtained from 102 experts to help develop climate adaptation measures in Korea. Experts were limited to those who have at least 10 years of experiences as a professor or doctor in the fields of health, water, forest ecology, land and coastal land, industrial energy, agriculture and livestock, and marine fisheries using climate change adaptation experts list from Korea Environment Institute. The 81 individual risk indicators combined climate drivers and damage types. For example, "increase in death due to heat waves", "increase in salinity due to drought", and "increased traffic accidents due to heavy precipitation", among others, can be climate change risk indicators in seven categories: health (HE), water (WA), forest ecology (ES), land and coastal land (LC), industrial energy (IE), agriculture and livestock (AG), and marine fisheries (MF). The reason why the existing research [21] was used as the basic data of this study is that the climate change risk indicators derived from existing research are data directly reflected in climate change adaptation policies in South Korea as scientific evidence and empirical data from experts in various fields related to climate change impacts. Some of the climate change risk indicators from existing study were not directly attributable to climate change, but there have been attempts to reflect only indicators with high agreement rate by experts from various fields. It included not only direct climate change impacts, but indirect impacts by human activity and natural disasters.
The risk type in this study were divided into human and natural system referring to [6]. For [6], human and natural system are regarded as subjects of exposure that can be affected adversely by climate change impact. In other words, they could be the subjects that should be protected from climate change impact. As a specific example, the settlement, infrastructure, human and human community were mentioned as one of human system. The final risk result of this study were classified into seven types (e.g., casualty, injury, disease, economic loss in the human system, extinction, damage and loss of value in the natural system) based on the concepts of risk [12,23,25,[30][31][32][33], so that risks could be quantified in accordance with the occurrence frequency reported in newspaper articles.
The major climate change drivers used for risk quantification were the six most common: drought, heavy snow, typhoons, heatwaves, cold waves, and heavy rainfall. In this way, it was possible to quantify the results of various climate change risks in human and natural systems arising from these six climate change drivers. For instance, while analyzing climate change impact news articles, we constructed the frequency of casualties, injuries, and diseases, based on 402 occurrences of drought. This result revealed the patterns and variables that influence climate change impacts; it may help to minimize future risks.
The risk indicators used in this study were selected from 73 of the 81 risk indicators described above existing study. The criteria for selecting the 73 required at least 10 matching frequencies per risk indicator assigned to each newspaper article in this study. This is because the climate change risk indicators that match up to 10 in 3098 newspaper articles mean that the probability of occurrence so far is very low. In addition to this, the risk indicators that are matched at a low rate should be excluded because they are unlikely to be identified as highly influential risk indicators in the one-to-all network analysis. In the Table 2 below, the 73 climate change risk indicators used in this study are presented, and the 8 climate change risk indicators excluded are also shown.   Threatens of dam stability and loss of regional assets from incresed frequency of storms ES24 Increased forest damage due to storm LC29 Increase in land use limit due to coastal erosion and flood risk

Newspaper Articles: Analysis and Parametrization
In this study, we drew on our previous study [18], which extracted 3098 newspaper articles related to climate change risks, using text mining to provide the basic data for a general risk assessment of climate change, a multi-risk analysis, and an estimate of the economic loss associated with each element of the multi-risk system described in this study. The 3098 newspaper articles were published over 24 years, since 1990. They were suitable for extracting data related to climate change risk in South Korea. In response to a previous study by [1], which revealed the local bias of newspaper articles, our data are based on 22 different national and regional newspapers (the Kyunghyang newspaper, Dong-A ilbo, Seoul newspaper, Hankyoreh, and Hankook ilbo as national newspapers, and the Gangwon do min ilbo, Gangwon ilbo, Gyeongnam do min ilbo, Gyeongsang ilbo, Gyeongin ilbo, Gwang-ju ilbo, Daejeon ilbo, Ma-il newspaper, Busan ilbo, Incheon ilbo, Jeonnam ilbo, Jeonbuk do min ilbo, Jeonbuk ilbo, Jemin ilbo, Chungdo ilbo, Chungcheoung today, and Halla ilbo as local newspapers). The reason we chose 22 newspapers was that the national newspapers chosen is highly reliable because it is included among the top 10 subscription rates in South Korea. In the case of 17 regional newspapers, we thought that they were not only highly reliable as the representative newspaper of six major districts (e.g., Gyeonggi province, Gangwon province, Gyeongsang province, Jeolla province, Chungcheong province, and Jeju province) in South Korea, but also could reduce regional variation. In addition to this, we tried to avoid political bias in newspaper articles by including progressive and conservative newspaper together.
The methodology of this study aims to derive a multi-risk perspective, based on data that match the risk indicators for each newspaper article. We have thus matched 81 risk indicators to each related newspaper article using the Python program, which allowed us to extract related newspaper articles by designating keywords related to each indicators to use in the data set of 3098 newspapers. For example, the main keywords in the risk indicators for "increase in mortality due to heat waves" include "heat wave" and heat island". In this way, we allocated related multiple-risk indicators to Sustainability 2018, 10, 1596 9 of 20 each newspaper article by using keywords that represented each of the 81 risk indicators. After an automatic matching process, a review was carried out to verify that the appropriate risk indicators matched the newspaper articles.

Network Analysis
Each of the multiple risk indicators listed in the newspaper articles provided basic data for the network analysis. Through a network analysis based on this constructed data, we investigated the relationship between 81 risk indicators. The connection strength and frequency values of this relationship were the basic data used to derive a multi-risk system in this study. We analyzed and applied the visualized data using the Netminer program, applying a linked list option that could analyze the risk indicators for each row.
In the network analysis, the key measurement indicators for structure identification included density analysis, concentration analysis, and centrality analysis [57]. Concentration analysis analyzes the degree to which an entire network is concentrated at a particular node, and measures the degree of connection, concentration, median concentration, closeness, and other relationships. Degree centrality is a measure of the degree to which a central node is centrally located within a network; it reflects the extent to which a particular node (risk indicators in this study) and other nodes (risk indicators) are directly related. In other words, the larger the degree, the more import the node is, and the formula to obtain degree centrality is as follows: Nodes with a high degree of centrality are located at the center of the network map, revealing a high degree of connectivity between the risk indicators in this study. Another measurement used in this study was closeness, which calculates the total number directly connected to other nodes. Here, the closeness measurement was replaced with the term "related risk frequency"; the higher this measurement, the more likely it is that climate change risks will occur at the same time. A network analysis and visualization were performed to analyze the strength of the relationship between risks. Nodes and links (edges) were presented in the visualization of centrality (Shown in Figure 1). The nodes represented each risk indicators, the analysis units of this study. The connecting links represented the links between climate change risk indicators. In other words, the many connection links in each risk (node) meant that the risk was likely to occur with other risks.

= =
Nodes with a high degree of centrality are located at the center of the network map, revealing a high degree of connectivity between the risk indicators in this study. Another measurement used in this study was closeness, which calculates the total number directly connected to other nodes. Here, the closeness measurement was replaced with the term "related risk frequency"; the higher this measurement, the more likely it is that climate change risks will occur at the same time. A network analysis and visualization were performed to analyze the strength of the relationship between risks. Nodes and links (edges) were presented in the visualization of centrality (Shown in Figure 1). The nodes represented each risk indicators, the analysis units of this study. The connecting links represented the links between climate change risk indicators. In other words, the many connection links in each risk (node) meant that the risk was likely to occur with other risks.

Research Process
In this study, we analyzed the relevance of keyword matching between newspaper articles regarding climate change impact and 73 climate change risk indicators referring to [21]'s proven climate change risk indicators. Through this process, 3098 newspaper articles related in climate change impact from [18] were matched with multiple relevant climate risk indicators, and the results of the study were designed in two major directions. First, national climate change risk trend was analyzed by analyzing the actual frequency and percentage of climate risk based on the climate change risk concept by [6]. Second, the climate risk indicators that are likely to occur at the same time with other climate risks were derived by analyzing the correlation between the climate change risk indicators. A detailed process of this is shown in Figure 2.

Research Process
In this study, we analyzed the relevance of keyword matching between newspaper articles regarding climate change impact and 73 climate change risk indicators referring to [21]'s proven climate change risk indicators. Through this process, 3098 newspaper articles related in climate change impact from [18] were matched with multiple relevant climate risk indicators, and the results of the study were designed in two major directions. First, national climate change risk trend was analyzed by analyzing the actual frequency and percentage of climate risk based on the climate change risk concept by [6]. Second, the climate risk indicators that are likely to occur at the same time with other climate risks were derived by analyzing the correlation between the climate change risk indicators. A detailed process of this is shown in Figure 2.
In this study, we analyzed the relevance of keyword matching between newspaper articles regarding climate change impact and 73 climate change risk indicators referring to [21]'s proven climate change risk indicators. Through this process, 3098 newspaper articles related in climate change impact from [18] were matched with multiple relevant climate risk indicators, and the results of the study were designed in two major directions. First, national climate change risk trend was analyzed by analyzing the actual frequency and percentage of climate risk based on the climate change risk concept by [6]. Second, the climate risk indicators that are likely to occur at the same time with other climate risks were derived by analyzing the correlation between the climate change risk indicators. A detailed process of this is shown in Figure 2.

Frequency Comparison According to Risk Final Type
Each of the 3098 newspapers providing basic data was a 100% match to at least one risk indicators, showing that climate change risk indicators and actual climate change impact data have a great correlation. However, the matching was weighted towards some risk indicators. For example, risk indicators such as "development negligence due to a shortage of water", "increased cost of protecting production facilities from heat waves and cold waves", and "increase in coastal environmental pollution due to overland pollution" were only matched about 10 times with each newspaper article, despite the average frequency of the risk indicators being around 100. This may, of course, mean that some risks are lower; it could also be due to the news coverage, which is affected by nation's politics, culture, and context.
After analyzing climate change drivers by text mining 3098 newspaper articles, it can be confirmed that the climate change impact of typhoons (1224) and heavy rain (1789) were higher than those of other drivers (see Table 3). Since some climate drivers overlapped, total frequency exceeded the 3098 newspaper articles. Overall, the risk frequency for the human system was overwhelmingly higher than the risk frequency for the ecosystem. This may reflect the nature of newspaper coverage, but it also shows how climate change risk is understood through data quantified by the degree of risk, depending on climate drivers. The highest level of risk frequency among the 7 types of risk: casualty, injury, disease, economic loss, extinction, damage, and loss of value, was economic loss due to heavy rainfall (1241); economic loss due to typhoons also appeared frequently (819). The overall risk frequency of typhoons and heavy rains was high, not only in relation to economic loss, but also in relation to casualties and injury. In addition to these, diseases caused by heat waves (49) and injuries caused by cold waves (41) were also outliers, compared to other risk patterns.

Analyzing and Comparing the Impact of the Multi-Risk Network
The analysis showed that the total relationship density between climate change risks was 0.5, which is a higher than normal level. The degree of connection was 0.7. This result showed that the network analysis of climate change risk indicators was organically structured, with linkages between risk indicators. The network analysis identified the mutual strength and correlation between risks; the top 30% of high risk connections were derived by analyzing the connection centrality of the total risk. Cases of high connectivity between climate change risks included, "increased mortality due to disasters", (38) "degradation and suspension of traffic facilities due to flooding", (32), "increased mortality rate due to short-term sudden weather change", (23) and so on (see Table 4). The occurrence frequency of each row in Table 4 reflects the matching frequency of each risk indicator to the 3098 newspaper articles.  We can therefore assume that a risk indicator with a high occurrence frequency is a highly probable risk in South Korea. On the other hand, a high risk value in centrality and its related risk frequency signify risk indicators that are likely to occur together during a climate event, thus representing a multi-risk system that will need to be managed in the future. Overall, risks with high occurrence frequency were positively related to centrality and related risk frequency as well-the higher the frequency of occurrence, the greater the likelihood that it would occur simultaneously with another risk. However, some risk indicators, such as "building damage due to strong winds", "destruction of river bank due to flood", and "damage to distribution facilities due to strong winds" ranked comparatively high in related risk frequency, although their occurrence frequency was lower than others (see Figure 3). Figure 4 is a visual representation of the interrelationships between climate change risk indicators. The risk indicator symbol in the figure corresponds to the content of Table 2: the method section in this study. The size of the nodes reflects occurrence frequency, and the link thickness indicates the mutual strength of climate risks. This figure not only identifies the multi-risks with high overall connection strength, but also the individual risks involved. For example, WA03 (shortage of water for daily use due to drought) is closely related to AG02 (increased damage to crops and livestock due to the spread of pests and diseases), HE25 (increased illness due to short-term sudden weather changes), WA07 (increased water supply gap between regions and classes due to drought), and so on. In particular, we deduce that risk indicator WA03 was highly correlated with WA07 by comparing the thickness of the link between two risk indicators. On the other hand, nodes (red circles) with no labels are risk indicators that have less than 10 matching risks per newspaper article, which are not connected to each other unlike the risk indicators in the middle in this Figure 4. In this way, the types of linked risks and the connection strength between risks can be figured out. comparing the thickness of the link between two risk indicators. On the other hand, nodes (red circles) with no labels are risk indicators that have less than 10 matching risks per newspaper article, which are not connected to each other unlike the risk indicators in the middle in this Figure 4. In this way, the types of linked risks and the connection strength between risks can be figured out.  Table 4 for a detailed description of each label in this figure.   Table 4 for a detailed description of each label in this figure.
Sustainability 2018, 10, x FOR PEER REVIEW 12 of 19 comparing the thickness of the link between two risk indicators. On the other hand, nodes (red circles) with no labels are risk indicators that have less than 10 matching risks per newspaper article, which are not connected to each other unlike the risk indicators in the middle in this Figure 4. In this way, the types of linked risks and the connection strength between risks can be figured out.  Table 4 for a detailed description of each label in this figure.   Table 2 for a detailed description of each label in this figure. The size of the nodes (red circles) reflects occurrence frequency, and the link thickness indicates the mutual strength of climate change risk indicators. Figure 5 shows the distribution of 17 major risk (see Table 4) indicators based on the areas of occurrence (six major provinces in South Korea) indicated by 3098 newspaper articles when analyzing text mining in this study. The results show that there is a difference in major risk occurring in each of the six regions, and indicate that the major risk occurring in Gyeonggi province and Gyeongsang province are relatively high compared to other regions.
Sustainability 2018, 10, x FOR PEER REVIEW 13 of 19 Figure 5 shows the distribution of 17 major risk (see Table 4) indicators based on the areas of occurrence (six major provinces in South Korea) indicated by 3098 newspaper articles when analyzing text mining in this study. The results show that there is a difference in major risk occurring in each of the six regions, and indicate that the major risk occurring in Gyeonggi province and Gyeongsang province are relatively high compared to other regions.  Table 4) for each of the six major province in South Korea including location reference. See Table 4 for a detailed description of each label in this figure.
In detail, risk indicators with high incidence in Gyeonggi province are HE18 (Increased mortality due to disasters, n = 400), HE24 (Increased mortality rate due to short-term sudden weather changes, n = 56), ES13 (Destruction of river bank due to flood, n = 42), LC09 (Degradation and suspension of traffic facilities due to heavy snow, n = 185), etc.
In case of Gyeongsang province, the frequency of occurrence of risk indicators such as HE19 (Increase in injuries due to disasters, n = 26), WA03 (Shortage of water for daily needs due to drought, n = 96), LC01 (Degradation and suspension of traffic facilities due to flooding, n = 262), LC06 (Decomposition and function degradation of river facilities such as embankments and bridges, n = 28) was highest than other regions. Jeolla province has a higher risk frequency of LC25 (Building damage due to strong winds, n = 32) than other regions. The climate change risk results, which are different depending on six regions, are meaningful themselves, further they can provide implications for regional adaptation planning to prepare for future climate change risks.

Risk Assessment
The background of this study and previous study [18] was that it has relied mostly on experts to construct and evaluate the risk indicators so far, which means that the risk assessment might be less objective. Therefore, this study conducted a risk assessment based on newspaper articles to suggest implications for constructing and using a risk indicator to ensure an effective response to climate change and complement objectivity as well. Compared to previous study, this study sought to elaborate the risk assessment. In other words, previous study has found major risk indicators with high matching frequency by simply searching for risk indicators per associated newspaper articles. The current study focused on finding risks that are more likely to occur simultaneously with climate change events using network analysis, which is the result of taking into account the importance on 'consequence' in consequence multiplication frequency, which is the concept of risk [6,14]. Moreover,  Table 4) for each of the six major province in South Korea including location reference. See Table 4 for a detailed description of each label in this figure.
In detail, risk indicators with high incidence in Gyeonggi province are HE18 (Increased mortality due to disasters, n = 400), HE24 (Increased mortality rate due to short-term sudden weather changes, n = 56), ES13 (Destruction of river bank due to flood, n = 42), LC09 (Degradation and suspension of traffic facilities due to heavy snow, n = 185), etc.
In case of Gyeongsang province, the frequency of occurrence of risk indicators such as HE19 (Increase in injuries due to disasters, n = 26), WA03 (Shortage of water for daily needs due to drought, n = 96), LC01 (Degradation and suspension of traffic facilities due to flooding, n = 262), LC06 (Decomposition and function degradation of river facilities such as embankments and bridges, n = 28) was highest than other regions. Jeolla province has a higher risk frequency of LC25 (Building damage due to strong winds, n = 32) than other regions. The climate change risk results, which are different depending on six regions, are meaningful themselves, further they can provide implications for regional adaptation planning to prepare for future climate change risks.

Risk Assessment
The background of this study and previous study [18] was that it has relied mostly on experts to construct and evaluate the risk indicators so far, which means that the risk assessment might be less objective. Therefore, this study conducted a risk assessment based on newspaper articles to suggest implications for constructing and using a risk indicator to ensure an effective response to climate change and complement objectivity as well. Compared to previous study, this study sought to elaborate the risk assessment. In other words, previous study has found major risk indicators with high matching frequency by simply searching for risk indicators per associated newspaper articles. The current study focused on finding risks that are more likely to occur simultaneously with climate change events using network analysis, which is the result of taking into account the importance on 'consequence' in consequence multiplication frequency, which is the concept of risk [6,14]. Moreover, by using text-mining technology to analyze newspaper articles, this study was able to identify national climate change risk patterns and even to quantify risks, based on the occurrence frequency of each (of seven) risk type. This data driven in this study can be used to help researchers recognize the conditions and patterns of future climate change risks. Deriving risk prioritization from risk assessment is a particularly effective way of understanding the impact of climate change and minimizing damage. It is also a key component of risk assessment and management, providing guidance for the implementation of appropriate risk reduction strategies and supporting the optimal allocation of available adaptation resources [40]. Most previous risk assessments have been conducted using specific models on a limited target or region [15,58,59]. These types of risk assessments (see Tables 3 and 4) are useful for understanding specific events or extracting accurate estimates, but they are difficult to use as a standard when comparing national or regional policies. Furthermore, topics related to climate change impacts are likely to require a multi-faceted approach, since not only individual cases, but also various sectors (e.g., agriculture, water, and infrastructure) interact, as has been pointed out in previous studies [14,60]. In response to these problems, the CCRA [14,36] determines a country's important risk indicators, evaluates the influence relationships and importance of each risk indicator, and reflects its results in the national adaptation policy. Topics covered in Committee on Climate Change, 2017 (CCRA 2017) include international dimensions of risk, such as food security, conflict, migration and humanitarian aid, as well as their inter-relationships [58]. Likewise, strategic responses to complex climate change challenges require a top-down approach, such as initially setting up and refining a risk indicator. Although it is important to establish the risk indicators as a first step, research to clarify relationships between risk indicators and to use them as basic data for risk quantification should be encouraged in future.
The innovation in this study is to use newspaper articles as a tool for climate change risk assessment. Although some may be skeptical about the value of newspaper articles as basic data, the vase amount of data that distinguish facts over time are highly likely to be used in the future. Moreover, newspaper articles about the impact of climate change can be divided into subjects and climate drivers; it will be possible to predict specific measures posed by interactive impact. However, as previous studies have pointed out [44][45][46], the subjective judgement and bias of newspaper articles must be taken into account. According to [2], the media's ability to define the issue of climate change does not take place in a vacuum. The mass media both shapes and is shaped by social, political, and economic forces. Other related studies [2,4] have also noted that a newspaper's political stance is critical, explaining the impact of the political frame on all of the articles in that newspaper. This study, in particular, has shown that some drivers or specific risk types (e.g., casualties; economic losses caused by a typhoon or heavy rain) occur very frequently. This type of risk involves the human system, and shows a remarkable divergence from risk frequencies in the natural system, which are relatively low. This shows that newspaper articles are not biased against climate events as a group, but that the damage pattern and frequencies center on specific objects. We know that not only the human system, but also the natural system, have been affected by climate change [6]. Moreover, risks in the natural system can cause secondary risks to the human system, which can be quite correlate with multi-risk driven in this study. It is therefore necessary to pay attention when assessing climate change risk using newspaper articles. To offset the subjectivity of newspaper articles, we used a survey of experts. Reference [61] has insisted that a multi-faceted approach is needed to deal with climate change issues. Uncertainty can be reduced by applying other beneficial methods, such as expert judgement, which can provide insight into the vulnerabilities of drivers associated with climate change. In [18]'s study, in particular, has argued that the judgement of climate change experts plays a crucial role in highlighting the problems of vulnerable groups and the secondary risks caused by climate change that cannot be deduced from quantitative data. Ultimately, the quantitative data and expert judgements presented in this study should complement each other to reduce the uncertainty and risk caused by climate change. For example, the results obtained in this study can be used as the primary data for evaluating the final risk indicators, and the second phase can use the expert evaluation. Expert elicitation or evaluation can assess the urgency, uncertainty, and the possibility of secondary risk for each risk indicators based on the primary derived value. In addition to this, risk indicators related with human system and natural system should be separately assessed on analyzing primary data considering that human related climate change incidents have been derived from many frequencies in the analysis of newspaper articles.

The Multi-Risk Approach and Its Policy Uses
The network analysis of climate change risk indicators and newspaper articles suggests implications for the future availability of multi-risk analyses and adaptation policies. The multi-risk indicators derived from this study represents risks that are likely to occur at the same time, when a climate change event occurs; these can lead to an increase in damage or a secondary risk of climate change in the human and natural systems. It is therefore necessary to intensively manage risk indicators that have a high correlation with each other or a high centrality shared between many risk indicators, ensuring that they are preferentially reflected in policy making. The issue of a multi-risk network and inter-connected relationships between risk indicators has been discussed in existing studies. Although, some decision models for multi-hazard and multi-risk assessment are being developed with the aim of providing stakeholders with a set of scenarios or alternatives [62], climate change has rarely been addressed, due to the complexity of its scope, measurement difficulties, and similar factors. Reference [37] have argued that the overall results of the review show that multi-risk approaches do take into consideration the effects of climate change; they mainly analyze static vulnerability by reviewing existing studies. Before we can overcome the limitations of the multi-risk analysis approach, it is necessary to establish as a basic framework a top-down approach for analyzing this kind of study deeply. In this study, the top-down approach, which assumes that policies set explicit aims and objectives that are directly translated into action on the ground, combines a content analysis of policy documents with policy maker interviews [63]. This study, therefore, can be used as a platform for further discussion of multi-risk issues, especially in a top-down approach related to a direct adaptation policy. It could be used to incorporate multiple perspectives and standards for a multi-risk approach.
Effective risk management should be built on a good understanding of all relevant threats affecting the target of interest, thus enabling decision makers and practitioners to develop efficient adaptation plans based on a robust prioritization of risk reduction measures [38]. Adaptation must be a priority for climate policy on the continent in this century, where "adaptation" refers to efforts to build resilience and reduce vulnerability to the impacts of climate change [64]. The purpose of this study is to investigate the proper selection and evaluation methods for risk indicators, which are important in climate change risk management. They should be taken into consideration in advance when developing climate change adaptation measures. Further, since national adaptation policy in South Korea are implemented based on the results of risk assessment [18], the risk assessment and adaptation policies are directly related.
This study's risk quantification and multi-risk network results can be linked with policy as follows. First, in implementing climate change adaptation measures by sector to reduce climate change risks, we should prioritize relevant adaptation policies by giving additional weight to climate change risks with high connection centrality, as well to indicators derived from expert elicitation and surveys. For example, policy makers can add importance and urgency on the adaptation policies, which are quite related to risk indicators driven in this study with high occurrence frequency and high centrality such as "Increased mortality due to disaster", "Degradation and suspension of traffic facilities due to flooding", and "Increased mortality rate due to short-term sudden weather change" (see Table 4). Second, it has been difficult to use risk indicators because of the hierarchical differences between climate change adaptation measures and climate change risk indicators when implementing policies.
The high level of risk identified by this study can help to further develop the risk indicators by establishing a hierarchy and integration process among climate change risk indicators.
The results of international organizations such as IPCC, OECD, and UNDP argue the importance of urban and regional countermeasures in adaptation policies [65]. This is because the unit where climate change occurs is a space, the adaptation policy which is not reflected in the urban planning is hard to be effective. It is vital cities to adapt to climate-induced risks [66]. In this context, ref. [67] presented an Adaptation Policy Framework (APF) as an alternative to harmonize the concept of sustainable development plan with climate change adaptation policy. The most effective way in APF is to develop an adaptation policy that reduces vulnerability from the perspective of urban planning. The specific measures of this adaptation policy are to match and manage the adaptive urban planning options that can reduce the regional climate change risks, and to prioritize the adaptation policies considering the characteristics of the region. For example, specific adaptive urban planning options to reduce LC01 (Degradation and suspension of traffic facilities due to flooding) derived from this study include water management facilities installation, embankment installation, rain garden installation, rooftop greening, green infrastructure planning, land use regulation in hazardous areas [68,69]. In addition to this, climate change risks vary depending on the vulnerability, exposure, and hazard in each region. One of the results on this study also showed that the frequencies of major climate change risk indicators for each of the six regions are different. Therefore, as proposed in previous studies [6,66,67,70], differentiating adaptation policies and urban planning according to their climate change risk results can effectively reduce climate change risk and minimize the budget for climate change adaptation policy as well.

Conclusions
This study has approached climate change risks from the top down in order to explore risk quantification and possibility of using a multi-risk approach to improve current climate change adaptation policies. This timely and relevant research will contribute to minimizing damage in situations where the policy flow related to climate change changes to risk management. This study can also supplement the limitations of the problem by relying on expert judgment when assessing the risks of climate change adaptation.
However, beyond the analysis of interrelationships within risk indicators, limitation of the present study is its inability to clarify the order in which incidents happen, so as to confirm whether each risk indicator occurred simultaneously or later. To compensate for this problem, future studies should select specific sites and analyze the risk indicators of climate change over a period of time in a comprehensive and timely manner, in order to identify the risk occurrence periods and their relationships within each sector. It can be an additional limitation that we did not subdivide climate change risk indicators using the concepts such as vulnerability, exposure, and hazard derived from previous studies. The risk assessment in this study can play a critical role in adaptation policy decision, elicitation, and prioritization, but there is a limit to how to quantify risks and provide concrete countermeasures considering secondary impact, indirect impact, etc. for each climate change risk. In future study, we intend to specify adaptation policy with urban planning measures to reduce climate change risk based on the major risk indicators in this study.
Author Contributions: All authors have contributed to the intellectual content of this paper. The first author, Y.K., developed the flow of this study, wrote most of manuscript, and gathered data including newspaper articles for 24 years of Korea. She was also responsible for all statistical analysis including text-mining analysis using R and network analysis. C.P. substantially contributed to the research design and wrote some of the manuscript and contributed to interpretation of all results and discussion.