An Overview of GIS-Based Assessment and Mapping of Mining-Induced Subsidence

: This article reviews numerous published studies on geographic information system (GIS)-based assessment and mapping of mining-induced subsidence. The various types of mine subsidence maps were ﬁrst classiﬁed into susceptibility, hazard, and risk maps according to the various types of the engineering geology maps. Subsequently, the mapping studies were also reclassiﬁed into several groups according to the analytic methods used in the correlation derivation or elements of the risk of interest. Data uncertainty, analytic methods and techniques, and usability of the prediction map were considered in the discussion of the limitations and future perspectives of mining subsidence zonation studies. Because GIS can process geospatial data in relation to mining subsidence, the application and feasibility of exploiting GIS-assisted geospatial predictive mapping may be expanded further. GIS-based subsidence predictive maps are helpful for both engineers and for planners responsible for the design and implementation of risk mitigation and management strategies in mining areas.


Introduction
Mining subsidence, a common mining-induced hazard, can result in severe damage to buildings, infrastructure, and the environment [1]. Numerous mines have been abandoned without the implementation of appropriate mine reclamation measures, and subsidence events have frequently occurred at underground mine sites worldwide ( Figure 1). Reliable subsidence predictions and mapping of future subsidence hazards in areas vulnerable to subsidence, based on continuous assessments and observations using accurate subsidence inventory data, represent a principal step toward effective mitigation of the risk of damage to property. Specifically, the presentation of engineering geological data in the form of an engineering geological map represents a useful tool for planners and developers in that the said maps indicate areas of potentially suitable and unsuitable land in relation to development. Numerous attempts have been made to assess mining subsidence using spatial information technology as it provides an effective approach to quantitative assessment at regional scales. A geographic information system (GIS) is a computer-based technology that enables the collection, management, analyzing, modeling, and presentation of geospatial data for a wide range of applications [2]. GIS can also be viewed as a computerized tool that provides a framework for solving geospatial problems [3]. Therefore, GIS is regarded as science or technology for spatial problem solving and can be used for scientific or engineering investigations, resource management, and development planning. In addition, GIS can be used to generate an engineering geology map. Because GIS can also be considered to be a form of a computer-coded digital mapping tool, it can be used to easily represent and combine factor maps to effectively derive the susceptibility, hazard, and risk indexes as well as perform effective modeling. Therefore, GIS plays a highly vital role in the entire process of mining hazard modeling and mapping production [4][5][6][7][8][9][10][11][12][13][14]. In addition, GIS-based geohazard mapping based on other concerned geospatial data sets can provide basic data for engineers and planners to help make spatial decisions.
Appl. Sci. 2020, 10, x FOR PEER REVIEW 2 of 24 (a) a wide-view (b) a lateral-view A geographic information system (GIS) is a computer-based technology that enables the collection, management, analyzing, modeling, and presentation of geospatial data for a wide range of applications [2]. GIS can also be viewed as a computerized tool that provides a framework for solving geospatial problems [3]. Therefore, GIS is regarded as science or technology for spatial problem solving and can be used for scientific or engineering investigations, resource management, and development planning. In addition, GIS can be used to generate an engineering geology map. Because GIS can also be considered to be a form of a computer-coded digital mapping tool, it can be used to easily represent and combine factor maps to effectively derive the susceptibility, hazard, and risk indexes as well as perform effective modeling. Therefore, GIS plays a highly vital role in the entire process of mining hazard modeling and mapping production [4][5][6][7][8][9][10][11][12][13][14]. In addition, GIS-based geohazard mapping based on other concerned geospatial data sets can provide basic data for engineers and planners to help make spatial decisions.
The scope of this paper was confined to reviewing only published literature concerning GISbased mining-induced subsidence mapping research that included the following three keywords (or concepts): mine or mining, subsidence, and GIS. As such, the articles that were not concerned with mining areas such as karst subsidence (e.g., depression, collapse, and doline), and urban land subsidence induced solely by groundwater withdrawal (i.e., not underground mining activities) were not considered. Numerous studies have reported on the detection and monitoring of ground subsidence in mine areas (here, subsidence inventory mapping) using remote sensing observation technologies (e.g., interferometric synthetic aperture radar (InSAR) and unmanned aerial vehicles (UAVs)). However, these are beyond the scope of GIS technology and are therefore excluded from this study.

GIS-Based Engineering Geology Maps
An engineering geological map refers to a type of a map that provides a generalized representation of all the key components of a geological environment in land-use planning and in the design, construction, and maintenance as applied to civil and mining engineering [15]. More recently, González de Vallejo and Ferrer [16] reported that geological engineering maps present the geological and geotechnical information for land-use planning, development, regeneration, and conservation, and to plan, construct, and maintain buildings, engineering structures, and infrastructure. According to Chacón et al. [17], engineering geology maps can be classified into inventory maps, susceptibility maps, hazard maps, and risk maps, as listed in Table 1. The scope of this paper was confined to reviewing only published literature concerning GIS-based mining-induced subsidence mapping research that included the following three keywords (or concepts): mine or mining, subsidence, and GIS. As such, the articles that were not concerned with mining areas such as karst subsidence (e.g., depression, collapse, and doline), and urban land subsidence induced solely by groundwater withdrawal (i.e., not underground mining activities) were not considered. Numerous studies have reported on the detection and monitoring of ground subsidence in mine areas (here, subsidence inventory mapping) using remote sensing observation technologies (e.g., interferometric synthetic aperture radar (InSAR) and unmanned aerial vehicles (UAVs)). However, these are beyond the scope of GIS technology and are therefore excluded from this study.

GIS-Based Engineering Geology Maps
An engineering geological map refers to a type of a map that provides a generalized representation of all the key components of a geological environment in land-use planning and in the design, construction, and maintenance as applied to civil and mining engineering [15]. More recently, González de Vallejo and Ferrer [16] reported that geological engineering maps present the geological and geotechnical information for land-use planning, development, regeneration, and conservation, and to plan, construct, and maintain buildings, engineering structures, and infrastructure. According to Chacón et al. [17], engineering geology maps can be classified into inventory maps, susceptibility maps, hazard maps, and risk maps, as listed in Table 1.
In terms of mine subsidence, an inventory map shows the location, type, abundance, activity, and date of past subsidence [18,19]. In the simplest case, this map type only shows the location of past subsidence. Because the past and present are keys to the future, an inventory map of the historical data can be used to predict the probability of future subsidence or to validate the prediction accuracy of the generated subsidence susceptibility map. Accordingly, the inventory map is the first step in any mitigation program. However, because an inventory map is mainly compiled through remote sensing observation technologies or field surveys, it is not considered a GIS analysis. Thus, published literature on inventory maps is excluded from this study. Table 1. Type and characteristics of engineering geological maps in terms of mining subsidence (modified from Chacón et al. [17]).

Type of Maps Characteristics
Inventory map · Locations, type, abundance, activity, date of (subsidence) events · Used to validate prediction accuracy of resulting (subsidence) susceptibility map Susceptibility map · Zonation of the relative spatial probability of future (subsidence) events · Ranks the stability of an area in categories that range from stable to unstable The basic concept of subsidence susceptibility includes the spatial distribution of factors related to instability processes; this concept is used for determining zones of subsidence vulnerable areas without any temporal implication. A subsidence susceptibility map shows where subsidence may occur via the ranks of ground stability of an area in categories that range from stable to unstable. Subsidence susceptibility has also been considered as an expression of relative hazard.
Subsidence hazard is defined as the probability of occurrence within a specified period and within a given area of a potentially damaging phenomenon [20]. The term 'Hazard' in this instance is different from what we commonly use for the word "hazard or in danger". The concept of a subsidence susceptibility map is different from that of a subsidence hazard map in that the latter includes zonations showing the annual probability (likelihood) of subsidence occurring throughout an area.
Although several different definitions of the term risk exist, one of the most frequently adopted definitions is presented by Varnes [20] as the expected number of lives lost, persons injured, damage to property, and disruption of economic activity caused by a specific damaging phenomenon for a given area and reference period. From this standpoint, it can be said that a subsidence risk map shows the expected annual cost of subsidence damage throughout the affected area and combines the probability information from a subsidence hazard map with an analysis of all possible consequences such as property damage, casualties, and loss of service [21]. The estimation of the risk associated with subsidence events can be computed (quantified) by multiplying the subsidence hazard index by the exposure intensity index and vulnerability index [22]. Table 2 summarizes the map type, analytical methods and techniques, verification/validation method, and spatial resolution in each study on mine subsidence susceptibility (or hazard) mapping in a GIS environment . Despite the aforementioned difference between subsidence susceptibility and hazard maps, all studies on the GIS-based mining subsidence assessment used the term subsidence hazard map in their title [23,25,28,30,31,[34][35][36]39,40], even though most of the papers used subsidence maps. Thus, the susceptibility and hazard studies were integrated and analyzed together in this paper.

General Procedure
In general, subsidence susceptibility mapping research comprises the following four stages: data processing, data analysis, data mapping, and validation ( Figure 2). Many GIS techniques, such as calculation, conversion, interpolation, map algebra, and other functions, are utilized in all these stages.

General Procedure
In general, subsidence susceptibility mapping research comprises the following four stages: data processing, data analysis, data mapping, and validation ( Figure 2). Many GIS techniques, such as calculation, conversion, interpolation, map algebra, and other functions, are utilized in all these stages. The first step is to collect the GIS data from a study area. A variety of geospatial data are incorporated into the GIS database, such as subsidence inventory maps (zoning or deformation), mine cavity (drift or panel) maps, topographic maps, hydrology maps, land use maps, road maps, building maps, geology maps, borehole data, and other site investigation data. Various subsidence contributory factor variables are selected from such GIS databases and processed as grid cell raster type thematic layers using GIS-based spatial analysis and interpolation techniques. In the process of compiling thematic maps, the size of the grid cell (spatial resolution) must be set to a range of 1 m to 10 m considering the extent and size of the study area. Numerous studies have adopted six to eight subsidence contributing factors. Subsequently, all the collected data are divided into training data and validation data to perform an unbiased analysis. In general, the ratio of the training area to the validation area is set to 70%:30%; however, the data split work does not require a theoretical approach.
The second step is to derive and interpret the correlation between past subsidence occurrences and the considered factors by analyzing and comparing the subsidence inventory map and each factor map (or all factor maps) spatially based on the analytical method or model. The analytical method or model includes probabilistic or statistical methods, fuzzy theory, expert systems, machine learning or deep learning, and an integrated approach. Correlation analysis is frequently performed outside the GIS environment, such as with the help of statistical analysis solution software or data mining software. However, when employing a theoretical approach in predicting future subsidence susceptibility, correlation analysis is not required.
The third step involves determining the subsidence susceptibility index (SSI) across the entire grid cell of the raster layer and generating a subsidence susceptibility map representing the relative possibility of subsidence occurrence in the study area in a GIS environment. Areas with high SSI The first step is to collect the GIS data from a study area. A variety of geospatial data are incorporated into the GIS database, such as subsidence inventory maps (zoning or deformation), mine cavity (drift or panel) maps, topographic maps, hydrology maps, land use maps, road maps, building maps, geology maps, borehole data, and other site investigation data. Various subsidence contributory factor variables are selected from such GIS databases and processed as grid cell raster type thematic layers using GIS-based spatial analysis and interpolation techniques. In the process of compiling thematic maps, the size of the grid cell (spatial resolution) must be set to a range of 1 m to 10 m considering the extent and size of the study area. Numerous studies have adopted six to eight subsidence contributing factors. Subsequently, all the collected data are divided into training data and validation data to perform an unbiased analysis. In general, the ratio of the training area to the validation area is set to 70%:30%; however, the data split work does not require a theoretical approach.
The second step is to derive and interpret the correlation between past subsidence occurrences and the considered factors by analyzing and comparing the subsidence inventory map and each factor map (or all factor maps) spatially based on the analytical method or model. The analytical method or model includes probabilistic or statistical methods, fuzzy theory, expert systems, machine learning or deep learning, and an integrated approach. Correlation analysis is frequently performed outside the GIS environment, such as with the help of statistical analysis solution software or data mining software. However, when employing a theoretical approach in predicting future subsidence susceptibility, correlation analysis is not required.
The third step involves determining the subsidence susceptibility index (SSI) across the entire grid cell of the raster layer and generating a subsidence susceptibility map representing the relative possibility of subsidence occurrence in the study area in a GIS environment. Areas with high SSI values (in general, expressed as red shading) show high subsidence potential; therefore, the SSI value can be used as a criterion to rank an area in terms of its subsidence hazard. Occasionally, the susceptibility map is also visualized along with past subsidence zones and nearby elements at risk, such as buildings or infrastructure.
The fourth step involves assessing the predictive capability (prediction accuracy) of the generated subsidence susceptibility map. The SSI enables the ranking of areas in terms of the likelihood of subsidence occurrence. To verify the accuracy of the SSI predictions, the generated subsidence susceptibility map is compared with a subsidence inventory map showing the distributions of actual past subsidence occurrences. The most common approach to the verification of prediction maps in subsidence contexts is to construct a cumulative frequency diagram (CFD) or a success rate curve (SRC) by comparing the generated map with the locations of past subsidence occurrences and subsequently calculating the area under the curve (AUC). This technique derives a prediction accuracy in the range of 50-100% of the proposed prediction model. The yielded prediction accuracy demonstrates the efficacy of the generated map in subsidence susceptibility predictions.

Literature Review According to Analytic Methods and Approaches
Subsidence susceptibility/hazard mapping studies can be classified into several groups according to their quantitative analysis method or approach used to derive and interpret the correlations between past subsidence and each contributory factor. Table 2 presents 16 analysis techniques applied to evaluate mine subsidence susceptibility.

Probabilistic/Statistical Approach
Probabilistic/statistical methods are utilized to predict the possibility of future subsidence by analyzing spatial data from an inventory of past subsidence events; this was done with the assumption that subsidence occurrences are determined by specific subsidence-related factors, and that future subsidence events will occur under conditions similar to those of past subsidence events.
Some studies have adopted the weight of evidence (WoE) method, which is a Bayesian probability model, to assess the susceptibility of mining subsidence. This method calculates the weight for each predictive factor based on the presence or absence of the training point subsidence units within the area of each binary predictor theme. Oh and Lee [23] assessed ground subsidence susceptibility at abandoned coal mine sites in Samcheok city in Korea using GIS and the WoE model. In this study, seven parameters were considered as subsidence-influencing factors, such as the depth of drift (cavity), distance from drift (cavity), geology, land use, slope, depth of ground water, and permeability. The spatial resolution (grid cell size) of the subsidence inventory map and contributory map was set to 1 m × 1 m. The WoE model was utilized to derive the correlation between the past subsidence binary value and each contributory factor value. The predicted map showed a 96.67% prediction accuracy based on the area using the ROC validation technique. Suh et al. [24] evaluated mining subsidence susceptibility using the WoE model by considering 3D complexed multiple mine drifts and estimated mined panels rather than employing 2D-based factors such as drift depth and distance to the mine drift to incorporate the complex effects of ground instability in the influential area. The SSI was calculated through the application of the WoE model, and it was shown on a map along with the damage level of buildings located in the mine area ( Figure 3). Validation using the CFD-AUC technique revealed that the suggested approach showed a 5.51% higher prediction accuracy than the case using 2D-based factors. The FR model has also been used to evaluate subsidence susceptibility. The FR is the ratio of the area where an event (in this case, subsidence) occurred in the total study area and is the ratio of the probabilities of an event occurrence to a nonoccurrence for a given attribute. The application process of the FR model for subsidence prediction is similar to that of the WoE method, and the calculation procedure is relatively simple. Choi et al. [25] predicted areas vulnerable to mining subsidence near abandoned underground coal mines. This study applied the FR model to determine each subsidenceinfluencing factor's relative rating (for each class of the factor), and then the coefficient of determination (R 2 ) between past subsidence occurrence and each factor was derived. Unfortunately, although there were many cases in which the coefficient of determination, which indicates the degree of relevance between ground subsidence and a specific factor, was less than 0.5 (low correlation), these factors were considered in the assessment of mine subsidence susceptibility. Son et al. [26] analyzed mining subsidence susceptibility by combining the FR model and radius of influence concept, which averages the FR value of each grid cell as well as surrounding cells within the radius of influence specified from the drift depth and the break angle. The resulting enhancement in prediction accuracy proved that this technique enabled the reinforcement of preceding statistical approaches by suggesting method averages. The verification result using the CFD-AUC technique ( Figure 4) revealed that the prediction accuracy of the suggested method (75.90%) was 8.31% higher than that of the existing method (the radius of influence concept was not considered) (67.49%). The FR model has also been used to evaluate subsidence susceptibility. The FR is the ratio of the area where an event (in this case, subsidence) occurred in the total study area and is the ratio of the probabilities of an event occurrence to a nonoccurrence for a given attribute. The application process of the FR model for subsidence prediction is similar to that of the WoE method, and the calculation procedure is relatively simple. Choi et al. [25] predicted areas vulnerable to mining subsidence near abandoned underground coal mines. This study applied the FR model to determine each subsidence-influencing factor's relative rating (for each class of the factor), and then the coefficient of determination (R 2 ) between past subsidence occurrence and each factor was derived. Unfortunately, although there were many cases in which the coefficient of determination, which indicates the degree of relevance between ground subsidence and a specific factor, was less than 0.5 (low correlation), these factors were considered in the assessment of mine subsidence susceptibility. Son et al. [26] analyzed mining subsidence susceptibility by combining the FR model and radius of influence concept, which averages the FR value of each grid cell as well as surrounding cells within the radius of influence specified from the drift depth and the break angle. The resulting enhancement in prediction accuracy proved that this technique enabled the reinforcement of preceding statistical approaches by suggesting method averages. The verification result using the CFD-AUC technique ( Figure 4) revealed that the prediction accuracy of the suggested method (75.90%) was 8.31% higher than that of the existing method (the radius of influence concept was not considered) (67.49%).

Fuzzy Theory
Choi et al. [27] applied certainty factor analysis and fuzzy theory to mining subsidence susceptibility estimations. Certainty factor analysis was used to estimate the relative weight of eight major factors influencing ground subsidence. The relative weight of each factor was then converted into a fuzzy membership value (0-1) and integrated as SSI using fuzzy combination operators to generate several coal mine subsidence susceptibility maps. Consequently, the fuzzy γ-operator with a low γ value and fuzzy algebraic product operator were specifically useful for ground subsidence prediction. This is a meaningful case study as an example of applying fuzzy membership functions (FMFs) and various fuzzy operators (i.e., fuzzy AND, fuzzy OR, fuzzy algebraic product, fuzzy algebraic sum, and fuzzy gamma operations) to mining subsidence studies. However, the previously mentioned use of the term 'subsidence hazard map' instead of the 'subsidence susceptibility map' in this study will need to be corrected.

Machine Learning and Deep Learning
Because of the recent growing interest in data mining technology worldwide, the use of machine learning methods for evaluating mining subsidence has become widespread in the last few years. However, although various deep learning techniques have recently been proposed, only limited techniques have been used in the field of mining subsidence.
An artificial neural network (ANN) method was utilized to predict ground subsidence in a GIS environment. Kim et al. [28] attempted to generate subsidence susceptibility maps using the ANN method with seven factor maps in a GIS environment. The weight of each factor was calculated using the back-propagation training method. Three-layered feed-forward networks (7 × 15 × 1 structure for each) were created using the MATLAB software package in this study. A subsidence susceptibility map was compiled using the determined weights of each factor. The verification results showed a 96.06% accuracy and exhibited sufficient agreement between the presumptive hazard map and the existing data on ground subsidence area. Similarly, Lee et al. [29] spatially predicted mining subsidence susceptibility using the ANN technique and eight contributory factors. The analysis procedures and methods of this study were similar to those of the previous study, except for the structure of the network (8 × 16 × 1). The results of the relative importance and weights of the factors revealed that the distance from the fault showed the highest value of 1.5477, followed by the geology factor in this case.

Fuzzy Theory
Choi et al. [27] applied certainty factor analysis and fuzzy theory to mining subsidence susceptibility estimations. Certainty factor analysis was used to estimate the relative weight of eight major factors influencing ground subsidence. The relative weight of each factor was then converted into a fuzzy membership value (0-1) and integrated as SSI using fuzzy combination operators to generate several coal mine subsidence susceptibility maps. Consequently, the fuzzy γ-operator with a low γ value and fuzzy algebraic product operator were specifically useful for ground subsidence prediction. This is a meaningful case study as an example of applying fuzzy membership functions (FMFs) and various fuzzy operators (i.e., fuzzy AND, fuzzy OR, fuzzy algebraic product, fuzzy algebraic sum, and fuzzy gamma operations) to mining subsidence studies. However, the previously mentioned use of the term 'subsidence hazard map' instead of the 'subsidence susceptibility map' in this study will need to be corrected.

Machine Learning and Deep Learning
Because of the recent growing interest in data mining technology worldwide, the use of machine learning methods for evaluating mining subsidence has become widespread in the last few years. However, although various deep learning techniques have recently been proposed, only limited techniques have been used in the field of mining subsidence.
An artificial neural network (ANN) method was utilized to predict ground subsidence in a GIS environment. Kim et al. [28] attempted to generate subsidence susceptibility maps using the ANN method with seven factor maps in a GIS environment. The weight of each factor was calculated using the back-propagation training method. Three-layered feed-forward networks (7 × 15 × 1 structure for each) were created using the MATLAB software package in this study. A subsidence susceptibility map was compiled using the determined weights of each factor. The verification results showed a 96.06% accuracy and exhibited sufficient agreement between the presumptive hazard map and the existing data on ground subsidence area. Similarly, Lee et al. [29] spatially predicted mining subsidence susceptibility using the ANN technique and eight contributory factors. The analysis procedures and methods of this study were similar to those of the previous study, except for the structure of the network (8 × 16 × 1). The results of the relative importance and weights of the factors revealed that the distance from the fault showed the highest value of 1.5477, followed by the geology factor in this case.
Park et al. [30] mapped ground subsidence hazards using an adaptive neuro-fuzzy inference system (ANFIS) model with different types of FMFs. The ANFIS method adopts neural network learning algorithms and fuzzy reasoning to map inputs into an output. This is similar to the fuzzy inference system in the framework of adaptive neural networks. The validation results showed similar prediction accuracy: 95.12% for the generalized bell-shaped MF model and 94.94% for the Sigmoidal2 MF model, which suggested that the choice of MFs was not important in the study. The authors of this paper concluded that the ANFIS technique showed an excellent predictive capability in subsidence prediction by combining the expert system (fuzzy inference) with the learning ability of the ANN method.
Lee and Park [31] constructed decision trees (DTs) using the chi-squared automatic interaction detector (CHAID) and quick, unbiased, and efficient statistical tree (QUEST) algorithms to analyze the relationships between past subsidence and related factors, as well as to map subsidence susceptibility near abandoned underground coal mines. Both subsidence susceptibility maps achieved by the DT model showed a better prediction accuracy (94.01% for the CHAID algorithm and 90.37% for the QUEST algorithm) than the FR model (86.70%).
Bui et al. [36] adopted four machine learning algorithms for land subsidence stability in mining areas. The four models include Bayesian logistic regression (BLR), support vector machine (SVM), logistic model tree (LMT), and alternate decision tree (ADTree). The seven subsidence-influencing factors were selected using the least square SVM technique. As a result, the most important subsidence conditioning factor for the study area was the distance to the lineament factor. This was followed by land use, lithology, lineament density, RMR, slope angle, distance to drift, and drift density. The validation results showed that the BLR model was the most distinguished model for subsidence management.

Comparative Studies
Some comparative studies have been reported in the assessment of subsidence susceptibility. Suh et al. [32] and Kim et al. [33] developed software called ArcMine as an extension toolbar in the ArcMap program that enables users to select up to 15 contributing factors as well as to select one to all methods of FR, FMFs, and the analytic hierarchy process (AHP) technique for comparative analysis and visualization of the mining subsidence susceptibility in a GIS environment. As a case study, the prediction accuracies of the two models were compared, such as the fuzzy model and fuzzy AHP model. The validation result showed that the fuzzy AHP model showed a higher prediction accuracy than the fuzzy model.
Kim et al. [34] assessed and compared ground subsidence hazards near an abandoned underground coal mine in Korea using FR and logistic regression (LR) models. This study compared the characteristics of the two models in the data processing step. The FR model is simple, and the application procedure is easy to understand. There is no need to convert attribute values to another format because the FR value can be used as a rating. Conversely, the LR model requires data to be converted to ASCII format for use in the statistical package and later reconverted to be incorporated into the GIS database. The verification results showed that the LR model (95.01%) had a better prediction accuracy than the FR model (93.29%) in the study area. The validation results revealed that both techniques showed a high predictive performance. Unfortunately, this study did not provide an interpretation of why the LR model showed a higher prediction accuracy than the FR model. Suh et al. [35] applied two methods (FR vs. FR-AHP) to generate subsidence hazard maps and compared two maps to determine the most accurate subsidence hazard map, which is one of the elements needed to compile a subsidence risk map. The CFD-AUC validation technique revealed that the FR model showed a higher prediction accuracy than the FR-AHP integrated model. Hence, the FR model-based subsidence hazard map was selected as one of the elements to map subsidence risk.
In terms of the four machine learning algorithms compared by Bui et al. [36] in predicting subsidence susceptibility, the ROC-AUC validation technique revealed that the BLR model produced a higher prediction accuracy compared to other applied models, even though the other models also had reasonable results. Similarly, Oh et al. [37] compared the predictive capabilities of different models in generating mining-induced subsidence susceptibility maps. The ROC-AUC validation technique revealed that the map generated by the logit boost model, one of the meta-ensemble machine learning models, showed the highest prediction accuracy (91.44%). This was followed by the logistic (88.92%), multilayer perceptron (86.76%), Bayes net (86.42%), and naïve Bayes (85.39%) models.

Integrative Studies
Unlike previous techniques, an integrative approach (or ensemble approach) has been proposed to support subsidence prediction and management of abandoned mine areas. Park et al. [38] proposed a new approach called the ensemble of several ground subsidence susceptibility maps. First, this study analyzed the correlation between past subsidence occurrences and related factors and computed the SSI of entire grid cells using three methods: FR, LR, and ANN. Second, the three SSI index maps were then used as new input factors and integrated using fuzzy ensemble methods to create better susceptibility maps. The validation result using the SRC-AUC technique revealed that the ensemble model was more effective in terms of prediction accuracy than the individual model.
Oh and Lee [39] also generated four different subsidence susceptibility maps of an abandoned coal mine using FR, WoE, LR, and ANN models and subsequently used the four maps as new input factors to compile an integrated ground subsidence susceptibility map. As a result, the integrated subsidence susceptibility maps that used the four new subsidence-related input factors showed a greater accuracy (96.46% for FR, 97.22% for WoE, 97.20% for LR, and 96.70% for ANN) than the individual subsidence maps (95.54% for FR, 94.22% for WoE, 96.89% for LR, and 94.45% for ANN).

Sensitivity Analysis
A sensitivity analysis for GIS-based mapping of the ground subsidence susceptibility near an abandoned underground coal mine was performed to determine the importance of extracted subsidence-related factors [40]. When comparing the combined effects of all factors except one for prediction accuracy, the distance from the lineament and the distance from the drift highly affected the occurrence of ground subsidence, and the groundwater depth, land use, and rock mass rating had the weakest effects.
Oh and Lee [23] considered seven subsidence-contributory factors in susceptibility mapping and the area ratio based on the ROC technique in calculating prediction accuracy as a validation. Then, five other subsidence prediction maps were created using five different combinations of the two factors among the seven factors. The combinations included the depth of the drift and slope (case 1), the distance from the drift and depth of ground water (case 2), the distance from the drift and land use (case 3), the distance from the drift, depth of ground water and land use (case 4), and using geology and land use (case 5). As a result, when all seven influential factors were considered, the prediction accuracy was the highest. On the other hand, the combination of depth of drift and slope showed the worst result of 76.48%.
Suh et al. [24] selected six subsidence-triggering factors to evaluate mine subsidence susceptibility and performed sensitivity analysis for two purposes: to examine how changes in prediction accuracy vary when the input factors are changed, and to generate an optimal prediction map by excluding factors that decrease the prediction accuracy. The prediction accuracies of the subsidence susceptibility maps based on different combinations of factors were calculated using the CFD-AUC technique. As listed in Table 3, the prediction accuracy increased when the slope gradient factor was excluded as an input parameter. This result indicated that the subsidence susceptibility map with five factors (without slope gradient) was the most appropriate model for the study area.

Vertical Displacement
In this section, unlike previous studies that mainly showed the susceptibility or hazard of mining subsidence as zoning (horizontal subsided area), several studies have predicted the vertical displacement (z-axis) based on past actual settlement amounts using regression equations or collapse theories. These studies usually provided a difference distribution map between observed subsidence and predicted subsidence. Vertical displacements can have positive and negative increments. The prediction methods used in this section describe the vertical displacement as subsidences.
The geographically weighted regression (GWR) method, which allows for spatial variability of subsidence factors, has been applied to map vertical displacement with subsidence zoning. In previous studies, most of the subsidence inventory data were binary type data, such as 0 (nonoccurrence) or 1 (occurrence). On the other hand, subsidence inventory data used in GWR techniques are mainly indicative of vertical displacement. Blachowski [41] predicted and mapped vertical displacement with subsided zoning (in three axis directions) within complicated mining conditions in Poland. Seven subsidence contributory factor data types and subsidence inventory data between 1886 and 2009 within the study area were used as inputs for the GWR method. As a result, cavity factors (thickness, inclination, and depth) and surface slope factor were identified as significant parameters. Using the regression equation between past vertical displacement and factor values, a hybrid map of subsidence for the entire study area was produced. The maximum predicted subsidence in these areas was calculated to be −10.5 m. Similarly, Cao et al. [42] adopted the GWR model and five triggering factors to predict amount of settlement in the Sanshandao area, Laizhou, Shandon Province, China. The accuracy of determining subsidence in the area used for validation was ±8.5 mm with a maximum calculated subsidence of −329.26 mm. The maximum subsidence predicted with the model for the seabed was −63 mm with a mean subsidence of −50 mm.
Conversely, a few studies predicted vertical displacement within mine areas using traditional theories or empirical equations. Hejmanowski and Malinowska [43] introduced the influence function methods called Knothe's theory-based subsidence prediction approach. This theory allowed for the input of various parameters (e.g., distance from the center of elementary exploitation, volume extracted in the exploitation part, radius of the range of main influence of mining activities, exploitation depth, angle of the main influence range, extraction coefficient, and others) to calculate the vertical displacement for the entire study area. Figure 5 shows a 5 m × 5 m raster grid-based map that demonstrates the surface subsidence that was measured and predicted, as well as the difference between the two values in 2005. Djamaluddin et al. [44] simulated the phenomenon of progressive movement distribution and damage to structures from large sequential coal mining in China using a new 3D GIS coupling model. This model combines theoretical methods of predicting subsidence over time using a stochastic medium concept involving the Knothe time function for basic governing equations to calculate progressive movement. Through coupling with GIS, this model can effectively and spatially model the vertical displacement and surface influential range according to the size and shape of the underground mining area. Unlu et al. [45] proposed a GIS-based integrated approach for ground deformation within coal mining basins. This approach integrates GIS data analysis, 2D finite element numerical modeling analysis, a comparison between predicted data, and highly accurate measurement data (i.e., GPS or InSAR). From the calibration process, the proposed approach provides more accurate results than those obtained from other classical subsidence prediction methods.
Appl. Sci. 2020, 10, x FOR PEER REVIEW 12 of 24 accurate measurement data (i.e., GPS or InSAR). From the calibration process, the proposed approach provides more accurate results than those obtained from other classical subsidence prediction methods.  Table 4 lists the subsidence triggering factors utilized to assess mine subsidence susceptibility within a GIS environment. Nearly 20 factors were considered in subsidence susceptibility mapping. These factors included mine cavity parameters (depth, thickness, inclination, proximity, density), the mining method, geological parameters (land use, land cover, proximity to fault, surface geology, RMR), dynamic loads (proximity to a road or railroad), hydrological parameters (surface runoff, groundwater, permeability), topological parameters (elevation, ground slope), and time parameters. This result indicated that mine subsidence can be caused by interactions among many factors, including internal geotechnical aspects, external dynamic loads, and human mining activities. The result also showed that the minimum number of the influential factors considered for each study was 3 and the maximum number was 11. Most of these studies adopted six to eight parameters to predict subsidence susceptibility.

Subsidence Contributory Factors
Distance to the nearest cavity, ground slope, and cavity depth factors were frequently used as input factors (in more than 80% of studies), followed by land use/land cover, geology, runoff/groundwater, and RMR factors (adopted in more than 50% of studies). Clearly, the influential factors frequently used in past studies have made substantial contributions to subsidence events. However, these studies have not necessarily considered that some factors are less relevant to ground subsidence because they are also affected by difficulties in collecting GIS data. Typically, the mine cavity density can be an important factor in predicting ground subsidence zoning because horizontal density or vertical overlap of underground cavities can be evaluated. In addition, the stability of the underground cavity is degraded over time, so the time factor can also be important. These problems require further research, such as numerical analysis or experiments of individual factors.  Table 4 lists the subsidence triggering factors utilized to assess mine subsidence susceptibility within a GIS environment. Nearly 20 factors were considered in subsidence susceptibility mapping. These factors included mine cavity parameters (depth, thickness, inclination, proximity, density), the mining method, geological parameters (land use, land cover, proximity to fault, surface geology, RMR), dynamic loads (proximity to a road or railroad), hydrological parameters (surface runoff, groundwater, permeability), topological parameters (elevation, ground slope), and time parameters. This result indicated that mine subsidence can be caused by interactions among many factors, including internal geotechnical aspects, external dynamic loads, and human mining activities. The result also showed that the minimum number of the influential factors considered for each study was 3 and the maximum number was 11. Most of these studies adopted six to eight parameters to predict subsidence susceptibility. Distance to the nearest cavity, ground slope, and cavity depth factors were frequently used as input factors (in more than 80% of studies), followed by land use/land cover, geology, runoff/groundwater, and RMR factors (adopted in more than 50% of studies). Clearly, the influential factors frequently used in past studies have made substantial contributions to subsidence events. However, these studies have not necessarily considered that some factors are less relevant to ground subsidence because they are also affected by difficulties in collecting GIS data. Typically, the mine cavity density can be an important factor in predicting ground subsidence zoning because horizontal density or vertical overlap of underground cavities can be evaluated. In addition, the stability of the underground cavity is degraded over time, so the time factor can also be important. These problems require further research, such as numerical analysis or experiments of individual factors.

Subsidence Contributory Factors
These various mine subsidence-influencing factors support that GIS can be effectively used to derive and analyze the influential factors of mine subsidence. The mine cavity density was calculated by applying a spatial density analysis technique to mine cavity polyline data. In addition, proximity factors such as distance to a mine cavity, geological lineament, road, and railroad are newly created through the distance analysis function of GIS. In the case of RMR or permeability factors, they are compiled by applying the geostatistical interpolation method of GIS to point-type borehole data. The elevation or slope of the topography was produced via the topographic slope analysis function. These geospatial influential factors enable the evaluation and prediction of ground subsidence from diverse perspectives.

Mine Subsidence Risk Assessment and Mapping
Many researchers have sought to assess the subsidence risk arising from mining activities. Table 5 summarizes the subsidence hazard factor, exposure item, vulnerability item, final risk item visualized in the map, and the spatial resolution of grid cells in the published literature investigated in this study. Most studies adopted buildings, land, road networks, and infrastructure as exposure factors to compute the risk arising from subsidence, whereas another study addressed agricultural crop areas at risk. In the case of a risk item as a final result, there were more studies showing a subsidence risk index and zone from a relative perspective than studies showing risk as an absolute number of loss or cost. In these studies, the spatial resolution (cell size) of the subsidence risk map was lower on average compared with those of the subsidence susceptibility maps.

General Procedure
As mentioned in Section 2, subsidence risk can be quantified by multiplying the subsidence hazard index, exposure intensity index, and vulnerability index. Consequently, in the process of creating the subsidence risk map, in addition to the subsidence hazard mapping procedure (Figure 2), generations of an exposure map and a vulnerability map as well as multiplication of the three maps is added, as shown in Figure 6. In this section, an exposure map shows an element at risk caused by mining subsidence, and a vulnerability map shows the element at risk. Strictly, the resulting subsidence risk map contains the expected annual cost or loss from an absolute point of view [20]. This is ascribed to the subsidence hazard already containing the concept of the probability of occurrence. Because subsidence risk mapping is a complex and difficult task, most of these studies lack a verification of the risk result.  Figure 6. Typical flow chart for assessing mine subsidence risk within a GIS environment. Figure 6. Typical flow chart for assessing mine subsidence risk within a GIS environment.

Buildings and Infrastructures
Several studies considered buildings or infrastructures as elements at risk associated with mining subsidence events.
Among them, some studies adopted the terrain deformation value as an element for estimating subsidence risk. Malinowska and Hejmanowski [46] assessed building damage in mining terrains based on a comparison between building strength and terrain deformation using a theoretical method in a GIS environment. This study predicted surface subsidence caused by coal and copper ore underground exploitation by employing the Polish approach from the influence function of Knothe's theory. Subsequently, the overall hazardous impact of mining on buildings (within a densely built up area) at the planned mining extraction site in selected periods of time was assessed by evaluating mining impacts on the surface, the resistance of objects in mining areas, and hazards in order. A fuzzy inference system was used to identify the damage classification (final form of result), such as structural buildings, to assess the impact of subsidence on surface properties. Mancini et al. [47] assessed salt mining activities related to risk at Tuzla (Bosnia and Herzegovina) by evaluating building density and intensity classes for four different hazards: deformation (sinking rate), water table rise, superficial fracture density, and deep fracturing. Using a multi-criteria decision analysis method including the FMF and AHP techniques, this study generated a final risk map with five severity classes over the area of abandoned salt mines, where care must be taken by urban planners and local administrators in their actions.
Conversely, a few studies used subsidence susceptibility maps as an alternative to subsidence hazard elements to estimate subsidence risk. These studies visualized the degree of subsidence risk due to mine development in five stages from very low to very high from a relative perspective. Tzampoglou and Loupasakis [48] mapped mining geohazard susceptibility and risk around the Amyntaio open-pit coal mine, West Macedonia, Greece, using GIS techniques. In this study, a subsidence susceptibility map was produced by the semi-quantitative method weighted linear combination (weight of factors and rate of each class within the factor). Subsequently, two thematic maps (i.e., land use and a road network) were considered as exposure elements to assess the subsidence risk. However, the vulnerability of the elements above was not considered in this study. Suh et al. [35] presented a GIS model to generate a subsidence priority map representing the relative risk of mine subsidence to buildings, which included residential and commercial buildings, playgrounds, gas stations, and other infrastructure. This study evaluated a subsidence hazard map based on the FR model. Subsequently, the subsidence hazard map was overlain with an exposure intensity map (building density) and vulnerability map (considering the distributions of buildings and depths of drift lines) to generate a subsidence risk map (Figure 7).

Agricultural Crop Area
A unique study that selected an agricultural crop area as an element exposed to the risk of ground subsidence was also found. Darmody [49] modeled agricultural impacts of longwall mine subsidence in Illinois, United States, using a GIS approach and a predictive model of agricultural soil subsidence sensitivity (SSS). The SSS model involves the integration of selected soil properties in a GIS to assign a subsidence sensitivity score to a given area. This study predicted annual crop yield losses of 6.8% at a suggested longwall mine area as a reference and revealed that mitigation of the affected areas would reduce average annual crop yield losses to 1.2% for the entire longwall area

Agricultural Crop Area
A unique study that selected an agricultural crop area as an element exposed to the risk of ground subsidence was also found. Darmody [49] modeled agricultural impacts of longwall mine subsidence in Illinois, United States, using a GIS approach and a predictive model of agricultural soil subsidence sensitivity (SSS). The SSS model involves the integration of selected soil properties in a GIS to assign a subsidence sensitivity score to a given area. This study predicted annual crop yield losses of 6.8% at a suggested longwall mine area as a reference and revealed that mitigation of the affected areas would reduce average annual crop yield losses to 1.2% for the entire longwall area (Table 6). Table 6. Predicted annual corn yields before and after longwall mining in the permit area (Darmody [49] Although the subsidence susceptibility values from this type of predictive modeling are not absolute and represent only a relative degree of hazard, they can provide a measure of subsidence initiation localities and assist in designing an effective subsidence warning management system (specifically, the area with high subsidence potential without past subsidence occurrences). Furthermore, a subsidence risk map enables decision makers to make appropriate choices and take measures that consider profit and loss. In this sense, these kinds of studies are important for both engineers and planners involved in the design and implementation of hazard mitigation and management systems in abandoned mine areas.

Discussion
The limitations and future improvements of GIS-based mine subsidence studies to date are described in this discussion.

Uncertainty of Subsidence Inventory Data and Time Effect
Input data (i.e., subsidence inventory map and influencing factor map) must be accurate and reliable because they can have a significant influence on the subsidence prediction result. However, numerous GIS-based subsidence studies investigated in this paper have data uncertainty problems from two perspectives.
The first is the diversity of mining subsidence types. In general, mine subsidence occurs in various forms such as continuous (trough) settlement, discontinuous (sinkhole) subsidence, cracks, curvature, and other phenomena. The cause and degree of damage for each type differ. Therefore, the diversity of the types of ground subsidence should be considered in the subsidence inventory data. However, most of the studies investigated so far only visualize the zoning information of ground subsidence. Therefore, it is necessary to classify the types of ground subsidence, analyze their cause, and predict the possibility of future occurrence. The second is the elapsed time effect (timing of the creation) of the underground mining cavities. According to published literature on the relationship between subsidence and time [1,50,51], the strength of the rock in the upper part of the underground cavity decreases owing to various causes, such as sagging caused by a loss of bearing capacity and changes in the groundwater level over time. In other words, even if it is safe, the probability of ground subsidence will increase over a long period of time (e.g., residual subsidence). Therefore, it is necessary to consider factors such as the elapsed time after the mine is abandoned in addition to the location data of the underground cavity in future studies. This part is also related to the development of 4D GIS that considers temporal aspects in 3D space.

Lack of Interpretation of the Cause of the Prediction Accuracy Difference (Applicability of Analytic Methods)
Most of the studies discussed in this review have presented the prediction accuracy for composed subsidence prediction maps based on specific analytical methods or approaches. In comparative studies [32][33][34][35][36][37][38], the prediction accuracies based on two or more methods applied to subsidence prediction mapping were compared, and some papers have compared their prediction accuracy with those of other studies to verify the superiority of the proposed analytical methods or approaches. However, it is difficult to find studies that faithfully present the interpretations or evidence for why method 'A' has a higher prediction accuracy than method 'B'. The use of method 'A' is often mentioned as more appropriate in the study area or in the subsidence prediction study because the prediction accuracy of method 'A' is higher. Alternatively, these studies may contain the interpretation that machine learning or deep learning techniques consider more factors than the general probability/statistical technique in which the characteristics of the contributory factors are not considered.
To enhance GIS-based subsidence research, it is necessary to determine the correct answer to this research gap. The prediction accuracy can be varied according to the reliability of the subsidence occurrence data, cell size, characteristics of the study area, influential factors, and the characteristics of the applied technique. However, it is crucial to carefully investigate the characteristics and applicability of each analysis technique applied to subsidence data. To this end, it is necessary to identify the cause of the difference in the prediction accuracies and the applicability of the analytical methods by producing subsidence test sites and GIS data, and by applying various analytical methods under the same conditions.

Subsidence Map for Evaluation or Prediction
Most mining subsidence studies have generated subsidence susceptibility maps over specific study areas by analyzing correlational information (values or equations) between maps of past subsidence occurrences and the contributory factors. Subsequently, they tend to present subsidence inventory maps overlain on generated subsidence susceptibility maps and simply analyze whether the locations of past subsidence occurrences matched regions with high subsidence potential. Certainly, such verification assists in the examination of the prediction accuracy and reliability of GIS analysis models used in these studies. However, this type of interpretation is simply 'evaluation' and does not encompass 'prediction,' Perhaps logically, regions of past subsidence are shown to have high subsidence potential in a subsidence susceptibility map irrespective of the method used, because it was used to derive the correlation. Moreover, the danger associated with regions of subsidence identified from the subsidence inventory map is already known. To extend the function of the generated map from 'evaluation' to 'prediction,' interpretation and discussion regarding areas with high subsidence potential, without past subsidence occurrences, should be addressed in future studies. These regions should also be included as locations where detailed site investigations or engineering measures are required to minimize the risks associated with ground instability.

Why Is Subsidence Risk Mapping Difficult?
Clearly, the number of case studies on subsidence risk investigated in this study is small compared with that of subsidence susceptibility/hazard mapping studies. A prediction of the geohazard (including subsidence) risk is still a highly difficult task. This difficulty can be ascribed to several reasons.
The first is because of difficulties associated with the analysis of the temporal probability of a subsidence event. As mentioned in Section 5.1, the probability of subsidence occurrence caused by underground cavities varies depending on the elapsed time after creation, and it is difficult to accurately determine the time of subsidence in rural areas. The second is because of the difficulties associated with the evaluation of subsidence vulnerability at risk. The vulnerability of an element at risk can be affected by the element's own characteristics (e.g., strength of the buildings) as well as the subsidence hazard. Hence, it is difficult to determine whether the degree of vulnerability is caused by a subsidence event or its own characteristics if a full investigation is not conducted. These aspects are similar to the difficulty of mapping landslide risk [52]. Third, the damaged area of mine subsidence may not be sufficiently wide. Compared with other geological disasters such as landslides, floods, and earthquakes, the influential ground area affected by mining activities is relatively small. For these reasons, the number of studies on subsidence risk mapping is considerably smaller than that of studies based on risk from other geological disasters.

Conclusions
In this study, the types and characteristics of GIS-based engineering geology maps were introduced, and various analytical methods and mapping cases for mine subsidence were reviewed. The typical procedure for mapping the mine subsidence susceptibility was presented, and various influential factors and analytic techniques considered in the evaluation, e.g., verification methods, grid cell sizes, and countries, were carefully investigated. Moreover, the method of mapping the subsidence risk was introduced, and the elements exposed to the risk and the type of final result were summarized. GIS technology has been proven to be effectively used in the process of processing and analyzing the mine subsidence from the geospatial perspective.
However, numerous studies on subsidence susceptibility mapping, which show the zonation of the relative spatial probability of future subsidence events, are incorrectly described as the subsidence hazard mapping in their title, which indicates the annual probability of a subsidence event. As such, researchers must pay attention to their choice of terminology when formulating their final map or title. In addition, as various deep learning techniques have recently been developed, it will be necessary to apply these techniques in mining subsidence prediction and to identify why a specific model is the most appropriate for subsidence prediction. Moreover, the published literature studies investigated in this paper do not include all studies on mining subsidence GIS. For example, numerous published literature sources where the GIS technique was applied to the mining subsidence were excluded because they are not included in the three map types according to the classification of engineering geological maps. In future research, it would be interesting to extend the scope of mining subsidence GIS research from various perspectives.
Although the subsidence susceptibility values from the various types of geospatial predictive models are not absolute and represent only a relative degree of hazard, they provide a measure of subsidence initiation localities and assist in the design of effective subsidence management systems. Furthermore, a subsidence risk map enables decision makers to select appropriate decisions and to take suitable measures from an economic perspective considering profit and loss. In this sense, these types of studies are helpful for both engineers and planners involved in the design and implementation of risk mitigation and management strategies in mining areas.
Funding: This research received no external funding.

Conflicts of Interest:
The author declares no conflict of interest.