Spatial Decision Support Systems with Automated Machine Learning: A Review

Abstract

Many spatial decision support systems suffer from user adoption issues in practice due to lack of trust, technical expertise, and resources. Automated machine learning has recently allowed non-experts to explore and apply machine-learning models in the industry without requiring abundant expert knowledge and resources. This paper reviews recent literature from 136 papers, and proposes a general framework for integrating spatial decision support systems with automated machine learning as an opportunity to lower major user adoption barriers. Challenges of data quality, model interpretability, and practical usefulness are discussed as general considerations for system implementation. Research opportunities related to spatially explicit models in AutoML, and resource-aware, collaborative/connected, and human-centered systems are also discussed to address these challenges. This paper argues that integrating automated machine learning into spatial decision support systems can not only potentially encourage user adoption, but also mutually benefit research in both fields—bridging human-related and technical advancements for fostering future developments in spatial decision support systems and automated machine learning.

1. Introduction

Advances in crowdsourcing [1], open data initiatives [2], and open-source standards [3] have made spatial data more publicly accessible. Spatial decision support systems (SDSS) store, manage, and process spatial and non-spatial data for important decisions, such as selecting business locations, placing traffic infrastructure, and implementing public-health policies [4]. However, many SDSS are not adopted by decision makers due to lack of trust, technical expertise, and resources [5,6]. Recently, automated machine learning (AutoML) has received attention from the research community and media. AutoML integrates automation and machine learning (ML) by generating models, with little human assistance, which perform well under certain requirements and computational budgets [7]. This reduces the effort and technical expertise required to process and model data, which accounts for a majority of the time spent on data analysis [8]. As leading technology companies released AutoML products in 2017 to 2018 [9,10,11], ML models became more widely used by non-experts and less expensive to implement. With the recent increase in accessibility to AutoML, resources for implementing and maintaining SDSS can be reduced, improving SDSS adoption by decision makers.

This paper provides a systematic review of AutoML and SDSS integration, which seeks to answer three research questions: (R1) What problems can both SDSS and AutoML solve according to recent research? (R2) How can AutoML be integrated into SDSS? and (R3) What are the challenges and opportunities of SDSS with AutoML to improve user adoption? Although there are existing review papers on AutoML and SDSS separately [4,12,13], review papers focused on the integration of both AutoML and SDSS were not found in the literature from an initial search on the topics of AutoML and SDSS together. This paper produces the following three research contributions to answer these questions: (C1) A systematic review that investigates recent methods, results, applications, and potential problems in SDSS and AutoML (C2) A framework based on recent literature for implementing AutoML in SDSS and (C3) A summary of key research opportunities and challenges of SDSS with AutoML in relation to user adoption.

Section 2 details the literature selection and review process. Section 3 briefly summarizes and analyzes the selected literature from Section 2, to provide an overview of important articles, topics, and trends. Section 4 reviews the selected literature for more in-depth background and theory of past and recent AutoML and SDSS theory, problems, and applications to answer research question 1. A framework of SDSS with AutoML is discussed in Section 5 to answer research question 2, along with key considerations, implementation challenges, and research opportunities to answer research question 3 as they relate to user adoption. Finally, Section 6 concludes the paper with a summary of the previous sections and future implications of research on SDSS with AutoML.

2. Methods

This paper used a two-step process to answer three research questions described in the introduction (Figure 1). This section details the process involving the gathering of relevant AutoML and SDSS research literature, then summarizing, analyzing, and discussing the gathered literature to answer the three research questions. An overview of the literature found from applying the process described in this section is available in Section 3.

2.1. Step One: Literature Search

The first step involved a search of recent AutoML and SDSS literature. Peer-reviewed journal articles were keyword searched (titles only) in 382 research databases (e.g., Scopus, arxiv, Web of Science, etc.) using Summon 2.0 between 1 January 2019 and 24 September 2022 [14]. These articles were then manually filtered by inspecting the title and abstracts for AutoML/SDSS-relevant articles that are review papers only. Since many recent literature review articles covered developments in AutoML [12,13,15,16,17] and SDSS [4,18,19,20,21] within the past 5 years, articles within three years were used to avoid outdated information and to focus on the most recent research. Thus, AutoML and SDSS articles that were not literature reviews or published earlier 2019 were excluded. The manually filtered articles (17 AutoML, 18 SDSS) in the past three years were used to also discover 101 additional supplementary references for both AutoML and SDSS topics using a snowball search strategy, which involved reviewing the manually filtered articles for major subtopics (e.g., feature selection, spatial clustering, machine-learning pipelines, etc.) to identify important supplementary references for AutoML and SDSS [22]. The purpose of identifying supplementary articles was to cover major SDSS and AutoML topics earlier than 2019, and was not limited to the past three years.

2.2. Step Two: Review and Discussion

The second step involved answering the research questions using the articles and relevant references gathered from the first step. Similar problems solvable by AutoML and SDSS were identified based on study objectives (e.g., predicting land-slide risk, simulating land use patterns, etc.) and summarized to answer research question 1: (R1) What problems can both SDSS and AutoML solve according to recent research? A general framework for SDSS with AutoML was developed to answer research question 2: (R2) How can AutoML be integrated into SDSS? Similarities in the methodology sections from the review papers in step one, and relevant references, were examined to identify AutoML and SDSS components. The AutoML and SDSS components were then connected based on general AutoML/SDSS approaches from the review papers/relevant references, and spatial problems identified from answering research question one. Finally, opportunities and challenges from the review papers and relevant references in step one were identified and discussed to answer research question 3: (R3) What are the challenges and opportunities of SDSS with AutoML to improve user adoption? Challenges and opportunities were found by comparing similarities and differences among the results, discussion, limitations, and other related sections/references.

3. Search Results

A total of 136 SDSS and AutoML articles were found for review. A total of 18 SDSS and 17 AutoML articles were used as primary sources to review recent important advancements, while 63 SDSS articles, 21 AutoML, and 17 AutoML/SDSS articles (articles having both AutoML and SDSS as topics) were found using these primary sources to serve as supplementary sources for reviewing foundational literature earlier than 2022. Primary sources (between 2019 and 2022) were important as they were made up of the most recent literature reviews to provide a detailed overview of significant research progress to date, while supplementary sources (earlier than 2022) were identified from primary sources to narrow down crucial sub-topics for AutoML and SDSS. The number of articles per year was steady between 1990 and 2018 (supplementary selection), before sharply rising after 2018 (primary selection for more recent articles) (Figure 2). Articles with both AutoML and SDSS topics were quite recent and seen only after 2020. Main keywords were focused on the topics of data, spatial/planning systems, machine learning, and models/analysis (Figure 3). Notable articles based on the number of citations are seen in Figure 4 and Figure 5, where citation data were found using the OpenCitations Corpus on 25 September 2022 [23]. These articles, grouped by topic and source, had a much larger number of citations compared to other articles in the same group. Notable primary SDSS articles had over 15 citations [4,19,24,25]. Notable primary AutoML articles had over 25 citations [12,26,27]. Notable supplementary SDSS articles had over 500 citations [28,29]. Notable supplementary AutoML articles had over 1000 citations: [30,31,32]. Notable supplementary articles with both AutoML and SDSS topics had far fewer citations due to recency and only had over 5 citations [33,34,35].

4. Review Results

Recall research question 1: (R1) What problems can both SDSS and AutoML solve according to recent research? This section addresses research question 1 by reviewing and summarizing the selected literature from Section 3 through a detailed overview of SDSS, AutoML, and their associated problems, methods, applications, and approaches in recent research.

4.1. Spatial Decision Support Systems (SDSS)

The term SDSS has been used since 1985 to describe software designed to support decision making by enabling users to analyze structured or semi-structured spatial problems for potential solutions [65]. Modern SDSS shifted from solution-centric software to human-centric frameworks which incorporated features and ideas from Geographic Information Systems (GIS) [66] and Planning Support Systems (PSS) [67]. SDSS are frameworks that incorporate a collection of tools designed to inform decision making involving spatially related problems, generally comprised of three components [4,18,19,24,36]: (1) spatial data (2) spatial information and (3) spatial knowledge.

The spatial-data component manages and processes data as input for the spatial-information component, which transforms the data into information (e.g., modelling, visualization) by organizing and presenting it for decision-making needs [21,68,69]. Approaches used in the spatial-information component include multiple criteria decision analysis (MCDA) [46], hot spot analysis [70], spatial regression [71], cellular automata (CA) [72], agent-based modelling (ABM) [47], and particle swarm optimization (PSO) [48]. This information acts as input for the knowledge component, which enables users to interact with and explore the information to produce knowledge [20,37,38,39,73]. Knowledge is used to support decisions or to help improve the data or information components [4,24]. Approaches for knowledge generation include participatory planning [19], citizen science [73], and geocollaboration [74]. Although the spatial data and information components focus on spatial data, they may also include supplementary non-spatial data. Each SDSS component contains several subcomponents representing more specific component features and functionality. General SDSS components, subcomponents, and their interactions are seen in Figure 6. The spatial-data component contains data-related sub-components (e.g., data collection, storage, access, documentation, processing, etc.) which serve as input for the spatial-information component. The spatial-information component contains several sub-components (e.g., monitoring, modelling, visualization, reporting, etc.) which transform spatial data into information (e.g., meaningful metrics during monitoring, plots/maps for visualization, automated textual summaries for reporting, etc.) which can be analyzed, interpreted, and, finally, transformed further into input for the knowledge component. The knowledge component contains action-related sub-components (e.g., communication, collaboration, exploration, etc.) which result in an action to change either the spatial data or spatial information components, and/or an action to implement or support a decision. These three components work together to produce actionable insights from the combination of data, information, and knowledge provided by SDSS.

Despite many studies in SDSS [4], challenges exist involving low user adoption [5,39,40] (e.g., awareness across disciplines, lack of practitioner acceptance/trust, expensive resources/training), evidence of usefulness [18,38] (e.g., proving practical utility/success for practitioners, added value vs. resources needed), adaptation [21] (e.g., balance between domain specificity and generalizability, application to similar contexts/problems), collaboration [36] (e.g., communication between non-technical and technical actors, translation of decision-making needs to models/tools), and interpretability [6] (e.g., excessive/complex information for non-technical users, transparency of processes/inputs/outputs). Discussion around the gaps between the research and practice of SDSS has remained an important topic, with many studies encouraging early involvement of, and collaboration between, decision makers, stakeholders, and the community [5,36,41]. Over time, SDSS research, largely focused on applications, case studies, and reviews, started to increase after 2004 and remained steady between 2010 and 2020, with a recent growth in studies related to urban science/analytics, smart cities/urban planning, and digital twins [75].

4.2. Automated Machine Learning (AutoML)

AutoML can be described as the automation of machine learning, a combination of two terms: (1) automation (Auto), to independently act, function, or operate without human intervention [76] and (2) machine learning (ML), a field of artificial intelligence (AI) focused on computer algorithms that can improve through experience [77]. Current AutoML approaches commonly involve optimizing components in the ML process (e.g., extraction/creation of features, tuning/creation of models) given constraints (e.g., reaching desired performances or time limits) [13]. These optimization approaches use metrics (e.g., accuracy, error) which determine the quality of the output components (e.g., features, models). Models include linear/logistic regression [78,79], naïve Bayes (NB) [80], decision trees (DT) [49], random forests [30], k-means clustering [32], support vector machines (SVM) [31], neural networks (NN) [81], and genetic algorithms (GA) [50]. A generic AutoML approach is seen in Figure 7, where two major processes (feature optimization and model optimization) are common amongst different AutoML approaches seen in the literature. The feature-optimization process converts data into a set of informative features which can be used to model a particular outcome, where these features are optimal based on a metric which measures the quality of each feature (e.g., how informative or varying each feature is). The model-optimization process then takes the optimized features and produces high-performing models based on the available models and associated model/optimization algorithm parameters provided. Model performance is measured by a metric of model quality which guides the model-optimization algorithm (e.g., accuracy or error between training and testing data).

Although AutoML has made AutoML more accessible to non-experts [7,82], there are issues in data dependency [12,26] (e.g., low data quality, unavailable data, data misuse), time and efficiency [17,83] (e.g., performance vs. acceptable running times, dataset sizes, search-space comprehensiveness), updates/reusability [13,42,43,84] (e.g., update existing model with new data, performance consistency, reproducible solutions), and interpretability [12,85,86] (e.g., why models perform better/worse or take certain actions). Closing the gap between domain experts/practitioners and ML specialists has recently been a topic of interest, as ML processes are increasingly automated, and applied to solve practical problems in industry [44]. Much of AutoML research has been focused on supervised learning, but recent research has diverged to tackle a larger range of ML problems such as unsupervised learning, time-series forecasting, and anomaly detection [87,88].

4.3. Spatial Problems in SDSS and ML

Spatial problems, commonly studied in SDSS, were not prevalent in AutoML research before 2020, when many studies focused on general problems, such as prediction and optimization, without considering spatial effects. However, ML applications to spatial problems are more common [89,90], and have recently been integrated into SDSS [21,91]. This section identifies spatial problems that have been studied by either SDSS or ML approaches to supplement the much smaller number of studies focused on both SDSS and AutoML. Since AutoML automates ML processes, it is relevant to review studies that use ML to solve spatial problems. A summary of spatial problems in SDSS and reviewed applications and approaches is shown in

Table 1. Reviewed spatial problems and approaches in SDSS.

Spatial Problem	Applications	Spatial Methods	ML Methods
Estimation	Land use classification Disease risk calculation Disaster risk prediction	MCDA Spatial Regression GW-ML	SVM RF NN
Optimization	Facility selection Delivery routing Infrastructure placement Traffic control	MCDA PSO	GA
Clustering	Crime hotspots Agricult./disease zoning Social-media analysis Travel analysis	LISA Hotspot analysis SaTScan	K-means DBSCAN
Simulation	Traffic control simulation Wildfire simulation Land use simulation	Cellular Automata ABM Custom models	RL
Insight	Risk factor identification Interactive exploration Data/model interpretation	Spat. Regression Webmapping Interactive models	Feat. selection Feat. importance Pipeline explore

, and examined in more detail in Section 4.3.1 to Section 4.3.5.

4.3.1. Spatial Estimation

Spatial estimation problems involve the calculation of unknown values at different locations, which encompasses spatial interpolation [92], prediction [93], and overlay [94]. These problems are solved to create surfaces from samples (e.g., kriging [29]), predict future values at different locations, and calculate values based on location. Recent examples include disease risk calculation [95], disaster risk prediction [96], and land use indicator creation [97] using MCDA approaches, spatial regression (e.g., geographically weighted regression (GWR) [98]), ML (e.g., SVM, RF, NN), and geographically weighted ML (GW-ML) (e.g., geographically weighted NN and RF) [99,100]. Spatial estimation solutions are often evaluated with error (e.g., root mean square error (RMSE), true negatives, sum of square error (SSE)) and accuracy (e.g., F1 score, area under the ROC curve (AUC), sensitivity, specificity) metrics which compare estimated values to true values from real-world data [51]. Challenges in spatial estimation problems involve the need for larger amounts of ground-truth data, and consideration of additional factors/variables/scales [93].

4.3.2. Spatial Optimization

Spatial optimization involves spatial placement [101] and routing [102] of entities. Solving spatial optimization problems helps to determine the more optimal placement of important facilities and infrastructure (e.g., site-selection or facility-location problems [52]), and efficient transportation paths (e.g., vehicle routing [103] and travelling salesman problems [104]). Recent examples include hospital facility selection [105], energy infrastructure placement [106], delivery routing [107], and automatic traffic control [108] using GA, MCDA, and PSO. In addition to error and accuracy metrics, spatial optimization solutions are evaluated with multicriterion [109] (e.g., weighted sums, sensitivity analysis) and multi-objective [53] (e.g., spread, hypervolume, convergence) metrics which consider important indicators from data, expert input, and search-space comprehensiveness. Challenges in spatial optimization problems involve temporal effects, multiple objectives, and computing efficiency [52,110].

4.3.3. Spatial Clustering

Spatial clustering involves grouping entities in space (e.g., local indicators of spatial association (LISA) [111], hot-spot analysis, density-based spatial clustering of applications with noise (DBSCAN) [112]) and time (e.g., spatiotemporal clustering [113], SaTScan [114], K-means [32]), where entities inside groups are similar and entities outside different groups are dissimilar [115]. These groups are used to identify interesting areas (e.g., zones with high crime [116] or natural resource potential [117]) and timespans (e.g., areas of disease transmission during specific times [118]). More recent applications involve clustering high-volume and -velocity spatiotemporal data (e.g., social media, crowdsourcing) [119,120]. Spatial clustering solutions are evaluated with statistical [121] (e.g., likelihood ratio, autocorrelation, significance) and similarity [54] (e.g., euclidean distance, pearson correlation) metrics. Spatial-clustering challenges include irregularly shaped clusters, high dimensional data, spatial relations/weights selection, resolution, object interactions, and visual vs. quantitative evaluation [113,122,123].

4.3.4. Spatial Simulation

Spatial simulation refers to imitations of real-world or hypothetical phenomena in space and time [124]. Spatial simulation enables analyses in cases where data is difficult to obtain (e.g., finer spatial/temporal resolutions, hypothetical/future phenomena) [47]. Approaches involve domain-specific models (e.g., crop-yield models [125], hydrodynamic-fluid models [126]), CA, and ABM. Recent examples include reinforcement learning (RL), ABM, and CA for simulating traffic-light control [127], wildfire spread [128], and sustainable urban growth [129]. Spatial simulation solutions are evaluated with domain-specific (e.g., total traffic delay [127], total crop yield [125], landscape composition/patch sizes [130]) metrics which are used to guide empirical observations. Challenges in spatial simulation include model validation, excessive complexity, disorganization, reproducibility, computing resources, and lack of theoretical basis [131].

4.3.5. Spatial Insight

Spatial-insight problems focus on interpreting and visualizing spatial data and model outputs, typically involving spatial regression [132], interactive interfaces [133], and maps [134]. Incorporating spatial-insight features in SDSS help data and models produce knowledge useful for decision making (e.g., graphical interface to interactively view and manipulate spatial data and models [135], web maps displaying spatial data or model results [136], coefficients representing variable effects in models [55]). Recent examples include spatial regression for identifying factors for reducing pollution [137], web GIS to interactively generate watershed models [138], and interactive visualization tools for exploring and analyzing AutoML pipelines [139]. Spatial insight solutions are evaluated with variable-based (e.g., feature importance [56], coefficients [28]), interpretability [57] (e.g., cognitive indicators, explanation indicators), and empirical approaches [140] (e.g., usability testing, controlled user surveys, user insight observation) to examine the effectiveness of interactive/visualization tools for producing useful insight. Spatial-insight challenges involve measuring usefulness, selecting/justifying appropriate presentation methods, handling big data/complexity, and personalization vs. generalizability [58,140,141].

4.4. SDSS with AutoML

Research related to SDSS with AutoML begun to emerge recently, after the year 2020. AutoML methods were applied to a variety of applications in the areas of agriculture (e.g., crop prediction [59,142,143], crop classification [60]), environmental science (e.g., environmental-impact assessment [144], waterlogging-risk estimation [145], water-storage estimation [33], water-potential mapping [146], meteorological forecasting [147], ocean behaviour prediction [148]), geology (e.g., oil-well placement [149], soil-roughness estimation [35], soil-moisture estimation [34], landslide risk estimation [61,150]), transportation (e.g., road health inspection [151]), and public health (e.g., violence rate prediction [62]). The majority of studies use a fusion of geospatial data sources with satellite imagery, sensors, and surveys being the most common and sociodemographic data being the least common. A summary of SDSS with AutoML approaches and applications is shown in

Table 2. Reviewed SDSS with AutoML approaches and applications (n = 17).

AutoML Approach	AutoML Method/Software	Data	SDSS Applications
Ensembling (n = 6)	H${}_2$O Extra trees class.	Satellite imagery UAV imagery Sensors Surveys Sociodemographic Simulations	Crop prediction Violence rate prediction Total water storage est. Landslide risk est. Soil estimation
Bayesian (n = 4)	Bayesian Opt. Auto-Sklearn Bayesian Nets MATLAB fitrauto	Satellite imagery UAV imagery Sensors Surveys	Crop prediction/classify Soil estimation Env. impact assessment
Neural Nets (n = 3)	NAS Deep Learning LSTM	Satellite imagery Vehicle imagery	Meteorological forecasting Road health inspection
Evolutionary (n = 2)	Fedot TPOT	Sensors Surveys	Oil-well placement Waterlogging risk est.
Other (n = 2)	Autogluon AutonML AlphaD3M	Sociodemographic Satellite imagerySurveys	Violence rate prediction Water potential mapping

.

The reviewed AutoML methods were grouped into four generalized approaches: (1) ensembling, (2) Bayesian, (3) neural nets, and (4) evolutionary. The most prominent AutoML approach was ensembling, which involves the combination of multiple algorithms to achieve better performance than individual algorithms [152]. This is followed by Bayesian approaches, which are based on the Bayes theorem and utilize past observations to guide future predictions [83,153]. Neural-net approaches involve the optimization of neural network architecture to build deep-learning models which achieve high performance [154]. Lastly, evolutionary approaches use algorithms which mimic natural-selection-based techniques on a population of models, such as mutation, reproduction, and selection, to find optimal evolved models that achieve better performance [155].

Reviewed AutoML methods and software include Auto-Sklearn, Tree-based Pipeline Optimization Tool (TPOT), H2O, Autogluon, Neural Architecture Search (NAS), and Alpha3DM. Auto-Sklearn uses a combination of Bayesian optimization, meta-learning, and ensemble construction to perform hyperparameter tuning and algorithm selection [156]. TPOT uses genetic programming to optimize and generate tree-based ML pipelines [157]. H2O uses random search and stacked ensembles to produce a final model which evaluates a diversity of candidate models, which, in some situations, are better than Bayesian optimization or genetic-algorithm-based approaches [158]. Autogluon distills ensembled models into individual models, using a data augmentation strategy based on Gibss sampling, to produce final models which are faster and, in some cases, more accurate than training individual models by themselves or ensembled models [159]. NAS automates the construction of neural network architectures to build deep-learning models with different search strategies, spaces and performance-estimation techniques, which can be applied to Bayesian Nets and long short-term memory (LSTM) networks [154]. AlphaD3M uses meta reinforcement learning with self play by modelling meta-data, tasks, and ML pipelines as states in a deep-learning model, which allows AlphaD3M to be faster than AutoML approaches such as Auto-Sklearn and TPOT, while being explainable with transparent ML-pipeline edit operations [160].

5. Discussion

Recall the two research questions: (R2) How can AutoML be integrated into SDSS? and (R3) What are the challenges and opportunities of SDSS with AutoML to improve user adoption? Section 5.1 answers research question 2 by applying concepts in the reviewed literature from Section 4 to form a framework for SDSS with AutoML, and discussing key considerations when applying the framework. Section 5.2–Section 5.3 answer research question 3 by identifying and discussing implementation challenges and research opportunities in SDSS with AutoML related to SDSS user adoption.

5.1. SDSS with AutoML Framework

Recent research in SDSS has incorporated ML algorithms and other models, which can be automated by AutoML, to solve spatial problems for transforming spatial data to spatial information. Applying the generic AutoML concept in Figure 7 to the SDSS components in Figure 6, AutoML can be integrated into SDSS by framing these spatial problems as optimization problems. Referring to the SDSS components in Figure 6, AutoML automatically processes spatial data into spatial information, having a role between these two SDSS components, but not the spatial knowledge component as it is dependent on human-driven processes (e.g., communication, exploration, collaboration). Given spatial problem x, potential solutions S, and metric Q (measuring how well solutions solve the spatial problem), AutoML can auto-approximate near optimal solutions $\tilde{S}$ for x based on metric Q within pre-defined constraints (e.g., time limits, desired performance). For example, a spatial estimation problem x could be to classify whether pixels at different coordinates are urban or rural land use. Metric Q can be a measure of how many pixels were predicted correctly based on groundtruth samples of urban/rural land use pixels, while potential solutions S can be a set of appropriate models (e.g., kriging, DT, NN) that can predict urban/rural landuse pixels. The near optimal solution $\tilde{S}$ is the most accurate model from S based on Q, using an optimization algorithm (e.g., GA, PSO) under constraints (e.g., max iterations/runtime). In MCDA, the potential solutions S can also be different weighing schemes. Referring to the AutoML components in Figure 7, a framework to integrate AutoML into SDSS is developed as seen in Figure 8, where AutoML automatically processes spatial data into spatial information by solving various spatial problems. In general, this framework requires three key considerations, which may also be applicable to geographic problem solving on a broader scale:

• Spatial problems: what are the spatial problem(s) to solve given the context and actors in decision making?
• Metrics: what metrics are appropriate for evaluating and measuring the defined spatial problem(s)?
• Potential solutions: with the given spatial problem(s) and metric(s), what are the potential solutions to solve the spatial problem(s)?

5.1.1. Key Consideration 1: Spatial Problems

Given the context and actors in decision making, spatial problems need to be defined to reflect decisions being evaluated [161]. Initially, determining the types of decisions being evaluated may help in defining the behaviour of decisions. The main types of decisions according to [162] are:

• Independent: decisions made by a decision maker with full responsibility and authority;
• Sequential interdependent: decisions made partially by a decision maker and partially by another party;
• Pooled interdependent: decisions made from negotiation and interaction among decision makers.

Then, three main steps related to the process of decision making as described by [163] can further aid in defining spatial problems:

• Intelligence: examination of spatial data to identify spatial problems that require decisions and have the opportunity for change;
• Design: determining possible and alternative decisions and developing approaches to evaluate and understand the decisions;
• Choice: selecting from the range of possible and alternative decisions after evaluating and understanding each decision.

After considering the type of decisions, the possible decisions, and approaches to selecting/evaluating the possible decisions, the spatial problem may be better defined as one or more (but not limited to) of the following general spatial problems as reviewed in Section 4.3:

• Spatial estimation: calculation of unknown values in space (e.g., prediction, overlay);
• Spatial optimization: optimization of entities in space (e.g., placement, routing);
• Spatial clustering: organization of entities in space (e.g., grouping, categorization, zoning);
• Spatial simulation: simulation of phenomena in space (e.g., physics, theoretical simulations);
• Spatial insight: interpretation and exploration of phenomena and entities in space (e.g., interactive maps, visualizations, plots).

Decisions can be simple or complex depending on the change desired from the decision and the evaluation approach designed. A simple decision may only require defining a single spatial problem. For example, the identification of crime hotspots for police patrols can be defined as a spatial-clustering problem. A more complex single decision may require defining one or a combination of different spatial problems. For example, analyzing the effects of health interventions may require the identification of intervention areas (a spatial-clustering problem) and simulating the effects of each alternative intervention on the intervention areas (a spatial-simulation problem). The considerations in this section are meant to be a starting point to help structure decisions as spatial problems, not as a definitive guide to do so as every context, actor, and decision-making process can vary drastically and there is not a solution for every situation [161].

5.1.2. Key Consideration 2: Metrics

After defining the spatial problem to solve, the metrics used to measure and evaluate potential solutions to each spatial problem need to be determined. It is important to consider the task needed to solve the defined spatial problems. For ML and statistics, the following tasks are common amongst studies [51,164,165]:

• Regression: estimation or prediction of continuous target values given other factors (e.g., calculating landslide risk, predicting number of traffic collisions);
• Classification: identification of discrete target values (groups or categories) given other factors (e.g., predicting landuse types, identifying building types);
• Clustering: organization of entities into groups or categories based on characteristics (e.g., identifying crime zones, disease areas).

Considering the tasks for solving spatial problems aids in identifying the metrics required to evaluate each spatial problem. Each metric has a particular purpose and is suited for measuring the performance of particular tasks. For example, RMSE and correlation coefficients metrics are used to evaluate regression tasks, while accuracy and F1 scores are used to evaluate classification tasks. As an example, metrics for the mentioned tasks above are provided in

Table 3. Example metrics for spatial problems and tasks.

Task	Metrics
Regression	Error, correlation coefficient, MSE, MAE, RMSE
Classfication	Accuracy, precision, recall, sensitivity, specificity, F1 score, AUC, ROC
Clustering	Euclidean distance, Rand index, entropy, purity, silhouette soefficient, Dunn’s index, Calinski–Harabasz index, homogeneity

. Ref. [51] presents a more comphrehensive collection of regression and classification metrics, while [54,166,167] provide more detailed overviews of different clustering metrics.

The choice of metrics should fit not only the defined spatial problem, but the behaviour and characteristics of the spatial data used. It is important to note that each metric has its own advantages and caveats [51,53,166]. For example, accuracy metrics are biased when the data contains unbalanced classes (e.g., 90% of the data is class A and only 10% is class B). This causes classification models to be high-performing if a large majority of the output from classification contains the dominant class. In this case, the F1 score is a more appropriate metric to account for class imbalances.

5.1.3. Key Consideration 3: Potential Solutions

When the spatial problems and associated metrics are defined, potential solutions to match these problems and metrics can be determined. The spatial problems and metrics create a structured goal for AutoML approaches to reach, where the potential solutions are often possible models or algorithms for AutoML methods to select from. The potential solutions need to accept input spatial data relevant to the defined spatial problems, while allowing the chosen metrics to measure outputs in a manner which is comparable across different potential solutions. Thus, a few considerations for potential solutions include [121,168,169]:

• Data size: how large or small the data are;
• Interpretability: whether the potential solutions need to be interpreted or simply produce outputs to be used (e.g., identifying important variables vs. prediction performance);
• Resource constraints: time, computation, and expertise constraints (e.g., runtime of models, training to interpret results);
• Update frequency: how often potential solutions need to be re-evaluated (e.g., new input data, new model/algorithm adjustments).

Similar to choosing metrics, the potential solutions generally have their own caveats and advantages [170]. For example, the performance of neural networks are reliant on the design of the neural architecture [154], while unsupervised models tend to perform better on larger datasets [169]. However, the difference between determining potential solutions and metrics is that, given the appropriate metric and adequate computing power, a comphrehensive search space can be defined and the choice of potential solutions becomes more flexible [171]. In more complex cases, potential solutions may also be allowed to be combined to form new potential solutions [172].

5.2. Implementation Challenges

This section discusses the implementation challenges of SDSS with AutoML as they relate to user adoption and its barriers. Section 5.2.1 discusses the issue of data-quality dependency, which creates a technical barrier to user adoption due to the need for expert model and data-management knowledge. Section 5.2.2 discusses model interpretability, where model outputs need to be explained to non-technical users (e.g., decision makers, the public), as this transparency largely influences SDSS user trust, a major user-adoption barrier. Most importantly, the usefulness of SDSS and the evidence to prove usefulness is a major challenge common among all SDSS, and may strongly affect a user’s need for a SDSS. This is discussed in Section 5.2.3.

5.2.1. Data Quality

SDSS and AutoML rely on data for producing models to generate useful information, which creates a technical barrier for users due to the need for domain expertise in data management and appropriate modelling. Data quality is an important factor in modelling, as it determines whether the data are appropriate for SDSS purposes or AutoML modelling. Data contains noise [12] (e.g., errors, incompleteness), varying levels of detail [173] (e.g., aggregate, scale), and risks [174] (e.g., misuse, ethics). As about 60% of the time is spent on data preparation [8], the challenge is to distribute resources to ensure that the data used is diverse (e.g., inclusive, transparent), representative (e.g., adequate coverage and detail), and reliable (e.g., minimal errors) for the intended purposes over time [26].

5.2.2. Model Interpretability

Predictions or actions from models are eventually explained to non-technical users in decision making (e.g., clients, society), which improves trust, transparency, and fairness [86]. Many reviewed AutoML studies measure model performance, but often do not focus on interpretability—why models perform better/worse or take certain actions [12,26]. Models in reviewed SDSS studies consider interpretability, but are often too complex for decision makers to use or communicate to stakeholders [24]. Improving interpretability leads to higher user adoption due to trust and useful knowledge from better communication. Another challenge is to balance available resources, model performance, and model interpretability to particular spatial problems.

5.2.3. Evidence of Usefulness

Evidence of usefulness, which influences user adoption, is often not focused on in the reviewed SDSS and AutoML studies. Without measures/examinations of usefulness, it is difficult to prove the added value of SDSS or AutoML in practice. This lead to issues such as difficulty differentiating SDSS implementations [6], low SDSS user adoption [18], inconsistent AutoML performance [13], and non-reusable AutoML models [84]. SDSS are also used in different application fields and domains (e.g., agriculture, forestry, environmental management), which have a variety of requirements and users which need adjustments and special attention on top of general SDSS [4]. An important challenge is to design methods to measure practical success by examining the utility of SDSS/AutoML implementations to real-world decisions and different domains and application fields—evaluating not only performance, but how SDSS with AutoML directly affects decision making.

5.3. Research Opportunities

This section discusses research opportunities to address the challenges in Section 5.2, where each opportunity covers one or more of the challenges (Figure 9). Section 5.3.1 discusses spatial AutoML, which was often not considered in the reviewed AutoML literature, but can reduce technical barriers to spatial modelling for SDSS users. The need for resource-aware approaches is identified in Section 5.3.2 to widen the variety of users in different resource-constrained environments. Section 5.3.3 addresses the problem of reusable and comparable systems, while balancing generalizability and specificity, to allow users to easily adopt different SDSS based on standards and collaborative communities across different domains and fields of practice. Lastly, a major barrier to user adoption involves the translation of complex inputs/outputs in SDSS, which can be improved with research in human-centered system design, which is discussed in Section 5.3.4.

5.3.1. Spatial AutoML

Many common AutoML approaches in the reviewed studies do not consider spatial data patterns when dealing with spatial problems. Although AutoML can automate non-spatial models, a majority of SDSS use spatial models and these present barriers in technical knowledge for users. If spatial patterns exist in the data (e.g., clustering and dispersion in space), then the assumption that observations are independent of each other is violated and the data has spatial dependency [175]. Spatial strategies (e.g., spatial sampling [63], localized models [176]) can be integrated into AutoML approaches for potential performance or efficiency gains, while spatial parameter problems (e.g., selecting neighbours or distance bands [64]) can fit in many AutoML optimization methods. One research opportunity involves incorporating spatially explicit approaches in AutoML for SDSS to improve modelling performance/efficiency and reduce arbitrary parameter selection without strongly affecting interpretability [25].

5.3.2. Resource-Aware Approaches

As SDSS and AutoML use various modelling and visualization approaches, the resources available (e.g., amount/quality of data, server processing power, data/domain experts) determine whether an implementation is feasible and practical for its intended purposes. The variety of SDSS users mean that their environments are different, where a major restriction is resource constraints when it comes to adopting SDSS (e.g., computing power, cloud infrastructure, modelling knowledge). In AutoML, the design of the search space (e.g., range of models/parameters) and optimization stopping criteria (e.g., iteration limits, reaching desired performance) are dependent on time tolerance, data available, and computing power [17]. Similarly, SDSS are dependent on hardware and software resources, but also include human resources (e.g., software developers, decision makers, consultants) which communicate and collaborate to design system features and purposes [36]. Another opportunity is to develop resource-aware implementation approaches for SDSS with AutoML by balancing the available resources and the desired results (e.g., data quality, model interpretability/performance).

5.3.3. Collaborative and Connected Systems

Many SDSS are difficult to reuse due to being developed for specific domain purposes [18], while AutoML models are difficult to reproduce due to varying search spaces and optimization approaches [12]. Reusability/reproducibility problems hinder collaboration and communication, as SDSS implementations and AutoML models do not follow standards for comparability (e.g., benchmarks, performance, features) and transferability (e.g., reuse for similar problems/different geographic area) [6,27,43,45]. Standardization of SDSS implementations and AutoML models enable SDSS with AutoML to be more easily compared/applied across various spatial problems [4], and shared across studies and organizations [36]. These standards enable different SDSS with AutoML implementations to be connected, which improves data and research transparency, availability, and interoperability (e.g., web platforms [177], programming interfaces [178], open data [69]). A third opportunity involves developing interoperable standards to connect SDSS with AutoML implementations in a network for sharing data/information/knowledge to reduce redundancy and repetition, while improving usability and collaboration between stakeholders, decision makers, and other actors.

5.3.4. Human-Centered System Design

Usability and interpretability are often overlooked in SDSS and AutoML research focusing on performance. When inputs and outputs are complex, barriers to translating information for useful decision-making knowledge hinder non-technical users (e.g., decision makers, stakeholders, policy makers) and lower user adoption [179]. However, a tradeoff between simplicity and accuracy exists, where improving the ease of use and transparency while minimizing complexity is desired [36,180]. SDSS with AutoML also need to balance domain specificity (e.g., custom solutions tailored to a particular spatial problem) and adaptability (e.g., flexible solutions applicable to a variety of spatial problems) [18]. A final opportunity is to strive towards human-centered system design principles and co-design for SDSS with AutoML, where careful considerations are made regarding user and spatial-problem characteristics, complexities, and interactions to enhance user adoption, experience, and practical usefulness.

6. Conclusions

This paper examined recent research for the integration of SDSS and AutoML to answer three questions: (R1) What problems can both SDSS and AutoML solve according to recent research? (R2) How can AutoML be integrated into SDSS? and (R3) What are the challenges and opportunities of SDSS with AutoML to improve user adoption? To answer question (R1), SDSS- and AutoML-related problems from recent literature were organized into five spatial problem categories (estimation, optimization. clustering, simulation, and insight) and summarized according to research application and spatial methods. A general framework for SDSS with AutoML was proposed to answer question (R2) by identifying and connecting general SDSS and AutoML components from selected research papers, where SDSS automatically processes data into information by solving spatial problems. To answer question (R3), challenges were discussed regarding data quality, model interpretability, and evidence of usefulness, while research opportunities were also discussed to address these challenges in relation to the user-adoption issue in SDSS. When implementing SDSS with AutoML, there is a distribution of available resources to maintain data with adequate quality/quantity for decision-making purposes, while ensuring models and systems are interpretable, comparable, and perform well in practice. One opportunity involves incorporating spatially explicit models, commonly used by SDSS, in AutoML research to help optimize, standardize, and compare models used in SDSS. Other opportunities involve developing standards, approaches, and principles for resource-aware, collaborative/connected, and human-centered systems. These developments support the goal of SDSS with AutoML, which is to aid decision making involving collaboration among various actors and different resource settings. As human-related (e.g., interpretability, usability, usefulness) and technical (e.g., reproducibility, reusability, comparability) issues arise in recent SDSS and AutoML research, integrating SDSS with AutoML incorporates technical aspects of AutoML (e.g., standardized pipelines/metrics) in SDSS research, while also incorporating human-related considerations of SDSS (e.g., solution complexity, scenario evaluation) in AutoML research. SDSS with AutoML not only helps improve SDSS user adoption, but mutually benefits SDSS and AutoML research by fostering approaches that consider both human-related and technical issues. The requirements, implementation, and user adoption of SDSS differ among different fields of study (e.g., environmental science, public health), and future work into exploring SDSS with AutoML for each field of study may provide further value for both SDSS and AutoML research.