Next Article in Journal
Uncertainty Analysis of Remote Sensing Pretreatment for Biomass Estimation on Landsat OLI and Landsat ETM+
Next Article in Special Issue
A Citizen-Centric Approach for the Improvement of Territorial Services Management
Previous Article in Journal
Multistage Cascade Predictor of Structural Elements Movement in the Deformation Analysis of Large Objects Based on Time Series Influencing Factors
Previous Article in Special Issue
A Data-Driven Framework for Walkability Measurement with Open Data: A Case Study of Triple Cities, New York
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Strengthening Participation Using Interactive Planning Support Systems: A Systematic Review

Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, P.O. Box 6, 7500 AE Enschede, The Netherlands
NEST (P) Ltd., Shankhamul-34, Kathmandu, Nepal
Author to whom correspondence should be addressed.
ISPRS Int. J. Geo-Inf. 2020, 9(1), 49;
Received: 18 December 2019 / Revised: 30 December 2019 / Accepted: 13 January 2020 / Published: 15 January 2020
(This article belongs to the Special Issue Geo-Information Science in Planning and Development of Smart Cities)


Interactive Planning Support Systems (PSS) implemented on a maptable are deemed suitable to support participatory planning processes. They are supposed to facilitate exchange of knowledge between stakeholders, consensus building among them, and group-learning processes. In this systematic review, based on 16 case studies using interactive PSS, we analyze how these have contributed to the goal of strengthening stakeholder participation. To this end, we first elicit details of the interactive PSS and the related participatory processes. In the second step, we analyze how and what the studies report, as the impacts on participation. Results show that tools and applications have become more sophisticated over time and goals of the studies changed from collaboratively designing interventions to observing and understanding how the application of such tools contributes to improved plan outcomes and group-based learning. All interactive PSS succeeded to facilitate intensive stakeholder collaboration. However, many studies lack a proper framework for investigating its impacts on participation and therefore assess these rather incidentally based on implicit assumptions. Thus, a significant outcome of this review is an evaluation framework, which allows the structural assessment of the impacts of interactive PSS on stakeholder participation.

1. Introduction

With the communicative turn [1], planning theory moved its focus from “a preoccupation with the distribution of material resources” to a “process of working out how to coexist in shared space” [2] (p. 219). Since then, collaborative planning, understood as an interactive and interpretative process involving a dialogue between stakeholders [2] (p. 221), has become the major paradigm guiding the practice of spatial planning and strategy making. However, the growing complexity of multidimensional planning tasks [3] calls for other forms of public participation, i.e., forms that go beyond what is legally required. These should enable “a multiple-way set of interactions among citizens and other players who together produce outcomes” [4] (p. 419). Closely related, Planning Support Systems (PSS), defined as “geo-information-technology-based instruments that are dedicated to supporting those involved in planning in the performance of their specific tasks” [5], are being implemented as instruments attuned to support specific planning tasks [6].
A particular type of PSS aiming to enable the collaboration of stakeholders are PSS developed for interactive surfaces, so-called interactive PSS. These interactive PSS are hardware systems in the form of a touch-sensitive table, a maptable, combined with geospatial mapping and analytical tools to support group processes [7]. Such interactive PSS provide a platform for discussing spatial problems or evaluating planned alternatives using a shared map interface to facilitate interaction among stakeholders [8]. Through their interactive nature and user-friendly interface a maptable might facilitate exchange of knowledge and preferences between stakeholders [9], consensus building among them [10], and group learning about complex problems [11,12], among others.
Since almost 15 years, various applications of such interactive PSS have been developed and implemented in case studies, often related to urban planning. These case studies, their specific contexts, tools implemented and participatory processes supported are often well reported. However, many studies focus rather on the evaluation of the usability [13] or user friendliness [14] of the PSS, i.e., the functioning of the tool [15], then on its usefulness, i.e., the impact of the use of such tools on the planning quality, though the usability of a PSS is rather a means than a goal in itself [16]. What is missing is a systematic review of case studies applying interactive PSS regarding their impacts on stakeholder participation.
We analyze in this review, based on case studies reported in interactive PSS publications, how such PSS have contributed to the goal of strengthening stakeholder participation. To this end, we first review existing interactive PSS and elicit details of the applications and cases studies, the PSS tools developed and the participatory processes they were used in. In the second step we then systematically analyze how and what the various studies report as the impacts of the use of the interactive PSS on participation, based on a framework for evaluating participation that we develop beforehand.
Accordingly, the paper is structured as follows: In Section 2 we provide an account of interactive PSS applications, tools and definitions and their use in stakeholder workshops. In Section 3 we develop a framework for evaluating impacts of interactive PSS on stakeholder participation. Section 4 elaborates the methodology applied in the review. Section 5 provides the results of the review analysis, i.e., the review of the interactive PSS applications, tools, and participatory processes (Section 5.1), and the reported impacts of interactive PSS on the various aspects and criteria of participation (Section 5.2). Section 6 discusses the results gained in the review before Section 7 concludes with key findings and further research needs.

2. Interactive PSS Enhancing Stakeholder Participation in Planning

Conceptual and technological research on maptables dates back to the turn of the century. Coors et al. [17] presented a first prototype of a virtual table composed of an interaction interface and 3D-GIS linked to it. In 2001, the MIT presented its Luminous Planning Table (LPT), a prototype developed as a “new form of information delivery that is direct and easily understood by professionals and laypersons” [18] (p. 195). Hopkins et al. [19] presented a so-called sketch-planning workbench implemented on a large-scale touch-sensitive screen that used gestures for drawing, annotating, and manipulating ideas during a collaborative sketch planning session.
The first case study of interactive PSS implemented in stakeholders workshops dates back to 2006 [20]. Since then various applications of interactive PSS have been developed and tested in various sectors of urban and regional planning, including sustainable urban development [11], climate change adaptation [21], energy transition [9], environmental health [12], and urban redevelopment [22], among others. Next to that, a couple of studies applying maptables in the field of disaster risk management can be found [23,24].

2.1. Interactive PSS Tools

An interactive PSS tool typically consists of a large-scale horizontal, touch-sensitive screen, that stakeholders can stand around and interact with, and a geospatial tool that enables the interaction (Figure 1). Sometimes a second, smaller screen is added to the setup to visualize the results of stakeholder’s interaction or other information, e.g., a 3D model. While over the last ten years, technological innovations have immensely improved the availability and affordability of large screen touchscreen as the key hardware for interactive PSS, suitable geospatial tools particularly made for interactive PSS applications are still hardly available. The development of interactive PSS tools is typically done by researchers for specific case studies and contexts, in some cases following a socio-technical approach [25] in order to increase their usefulness [15].
Proven benefit of an interactive PSS compared to a desktop PSS is that the maptable has a positive influence on working styles and group dynamics and thereby allows an enhanced interaction between stakeholders. Therefore, such interactive PSS are particularly suited to support participatory activities within a planning process. Dias et al. [26] describe the use of such interactive PSS in participatory planning processes as a promising way to bridge the gap between the creative design process of architects and urban designers and the more analytical process of planners. Ryall et al. [27] report the use of such applications “less intimidating, and often more playful than a traditional computer desktop environment”. Doeweling et al. [28] compared the use of classical paper maps, desktop GIS and interactive maptables in the context of stakeholder collaboration during crisis management. The authors found that a maptable-based solution can provide significant advantages when multiple users with different roles collaborate in sense making and planning tasks. Finally, the horizontal setup of a maptable revealed significant benefits for group collaboration compared to a vertical setup of the same device [29,30].

2.2. Planning Processes and Interactive PSS Workshops

A typical planning process can be structured into the phases of intelligence, design and choice [31]. The intelligence phase often comprises a baseline and problem analysis, in the design phase typically alternative plans or solutions are designed, while in the choice phase a decision is made on which plan alternative to implement [32]. Theoretically, stakeholders might be involved in any or even multiple of these phases of a planning process, e.g., by contributing the problem analysis, designing alternative plans, or evaluating the different alternatives [33].
The application of interactive PSS tools is typically done in PSS workshops [34] that often relate to a specific step or task within such a planning process. Two types of workshops can be distinguished here, controlled experiments and close to real-world stakeholder workshops. In controlled experiments, tasks often involve students playing a role of stakeholders [34]. Such experiments allow to test specific questions with respect to the usability of the tool, a comparative analysis of different tools [28,35], or tools being tested under changing conditions or with different user groups [36]. Moreover, regularly larger samples for statistical analysis are achieved.
On the other hand, in close to real world stakeholder workshops, tests provide richer information on their added value for the planning process and outcomes. However, those workshops are more complex to set up and implement, and include a certain level of uncertainty. Ideally, controlled experiments with a PSS are conducted prior to using it in stakeholder workshops. Finally, both types of workshops benefit from a high degree of facilitation, including a moderator steering the discussion and a chauffeur supporting the use of the PSS [37].
Methods for the evaluation of the use of interactive PSS in stakeholder workshops range from observations, interviews with participants, surveys based on questionnaire, to audio or video recording of stakeholder sessions. Often various methods are combined in order to verify the consistency of findings based on multiple sources via triangulation. Janssen et al. [38] conducted participant’s surveys before and after the workshops in order to explore how perceptions and preferences changed throughout a planning session.

3. Impacts of Interactive PSS on Participation

In order to analyze to what extent interactive PSS tools and applications have contributed to the goal of strengthening participation during PSS workshops, a framework for evaluating participation is needed. Such framework can be built from two partly separated scientific debates: First, the evaluation of public participation, which is an essential element of planning practice; second, the evaluation of PSS in the context of planning support science [39].
The evaluation of public participation within a planning context essentially looks at the effectiveness [40] or success or failure [41] of the participatory activities and processes. Gunton and Day [42] (p. 9) identified four key evaluative criteria for assessing participatory practices, which are: “(1) Success in reaching an agreement, (2) efficiency of the collaborative process relative to alternative processes, (3) satisfaction of stakeholders with the process and outcome, and (4) achievement of other social capital benefits such as improved relationships among stakeholders and enhanced stakeholder skills and knowledge”. More elaborated evaluation frameworks were developed by Rowe and Frewer [43], Laurian and Shaw [41], Brown and Wei Chin [40], and Mannieri and Talo [44]. Commonly, these frameworks distinguish between process criteria, that refer to the effective construction and implementation of a participatory activity, and acceptance or outcome criteria that refer to the broad range of results obtained from a participatory activity [40,43]. The frameworks include a number of—partly similar—process and outcome criteria. Depending on the goals of an individual participatory activity different sets of criteria might be taken into account, because “each of these criteria suggests a different perspective on what ‘successful’ participation means” [41] (p. 296). Moreover, these criteria are not mutually exclusive, because a participatory process might “seek to promote different goals simultaneously” [41]. Finally, these criteria are developed and used independent of the individual participatory methods, e.g., public hearing, consensus conference, focus group discussion.
The evaluation of the use of PSS in participatory workshops has focused for long time rather on the usability of the PSS tools [13,45] than on its impacts on the participatory process. Only since 2010 the latter has gained attention in the scientific debate on PSS (e.g., [7,8,14,15,16,46]. However, studies that evaluate the impact of the tools on the quality of the planning process and outcomes adopt different assessment frameworks, such as performance [16], added value [7,46], or effectiveness [47], all of them being based rather on implicit assumptions than on empirically grounded outcomes [15]. These frameworks likewise distinguish process and outcome criteria, but a thorough operationalization of the criteria is often lacking. Instead, scholars investigate potential correlations between the usefulness of a PSS and certain explanatory variables manifested in the usability of the particular PSS tool in order to determine options for improvement of the PSS. For example, te Brömmelstroet [14] has investigated the correlation between the usefulness of a PSS and its user-friendliness.
The evaluation framework developed for this study (Table 1) integrates dimensions and criteria from both scientific debates. It distinguishes criteria to evaluate the participatory process as well as those to evaluate the outcomes achieved through the participatory process. Both types of evaluation criteria are meant to evaluate participatory activities conducted in a PSS workshop context.
Process criteria include on the one hand criteria assessing the quality of the communication and collaboration between stakeholders during the PSS workshop and on the other hand whether and how social learning, understood as a learning through group activities [48], happens during the process. A participatory activity enabling a good communication and collaboration between stakeholders supports what Laurian and Shaw [41] call a democratic process. Such a process should ideally be transparent ([40,41], fair [40,41], inclusive [40,41,44], and efficient ([7,16]. The two criteria of a process being inclusive and fair are closely related to each other, though the latter is sometimes overlooked in participatory practice [41]. Inclusive means that all participants are given equal opportunities to attend a workshop [44]. Fair means that all participants are given equal voice or standing [41], the dialogue is free from bias, no group or single participant is dominating the discussions, and all participants are respectful with each other [44].
Criteria for social learning during a PSS workshop evaluate the development and increase of knowledge and understanding [44] of the participants during the participatory PSS process. Different levels of social learning are to be distinguished here. Awareness raising occurs when participants are informed about issues and stakes at hand [41] and they increase their knowledge about these [9]. Single and double loop learning occurs when repeated experiences shared in group dialogues help participants to detect a mismatch between intentions and consequences [52]. Single loop learning is defined when participants are challenging their behavior to address a challenging situation, while double loop learning refers to participants reflecting underlying assumptions and values [53].
Outcome criteria are structured into issue related outcomes and social outcomes [41]. The latter is frequently neglected in PSS workshops, probably due to their singular, often project-related character. Issue related outcome criteria evaluate the broad range of results obtained from a PSS workshop. They comprise the creation and/or integration of knowledge, such as tacit, contextual-embedded, and codified knowledge [54], whether there is an enhanced understanding of the problem [12], whether a consensus is achieved, and whether the quality of the decision is improved [41]. The last one also includes the aspect of novel ideas or issues being developed (e.g., new alternative plans or interventions), that te Brömmelstroet [14] mentions. Social outcome criteria measure whether PSS workshops result in capacity building, and increased trust and social networks [41]. They include creation of commitment and ownership among the participants [16,50], the development of a mutual understanding of each other’s perspectives and issues [14,16,40,41], the initiation of community building, e.g., new collaborations or improved social cohesion [51], and whether participants are satisfied with the outcomes of the workshop [14,16,40,41].
Though separated in the framework, process and outcome criteria are closely related, because they likely affect each other [51]. For instance, achieving a common and good understanding of the problem is a prerequisite of achieving a consensus. Likewise, a fair and inclusive process is not only a prerequisite for consensus building but to a certain degree also for achieving a common and mutual understanding.

4. Review Methodology

The review is based on the analysis of studies describing interactive PSS that are used for facilitating stakeholder collaboration in urban planning related participatory activities. Figure 2 depicts the entire review methodology.
The review includes tools and applications published in journal articles and conference proceedings. If various articles refer to the same interactive PSS tool, they are considered as one study. The search strategy for finding publications included two steps. First, we searched for publications from the year 2000 to 2018 in databases using relevant combinations of keywords plus commonly used synonyms. We then screened the obtained long list of publications and selected papers based on predefined selection criteria. Scopus was used as the search database, because it also includes publications in conference proceedings, with the following search terms:
  • Keyword = maptable or one of the following synonyms: touchtable, tabletop, touch user interface, tangible user interface, tui, plus different spellings of these terms such as map table;
  • AND
  • Keyword = urban planning or one of the following related terms: spatial planning, strategic planning, urban design.
In total 58 papers were found in Scopus. After reading the abstract of these papers, a short list of papers to be included in the review was compiled based on the following selection criteria:
  • Focus on stakeholder participation in spatial planning in general, i.e., planning related activities at any scale (national to sub-local),
  • Use of the maptable to facilitate the participatory process,
  • Real-world case study, i.e., applied in a real context, with real stakeholders, and addressing a real or at least realistic problem setting, i.e., a typical contemporary planning issue,
  • Peer reviewed and published articles, book chapters, or conference papers,
  • Interactive PSS applications, tool and workshop described in sufficient detail, so that the necessarily details for all criteria can be elicited.
The following table presents the 16 tools and papers included in the study (Table 2). All studies describe the applications of interactive PSS in stakeholder workshops to support steps in a planning process. Some studies made use of multiple case studies. Study ID 6 included one case that was not truly supporting a group process, because stakeholders had to use the table one by one.
The analysis of the studies is structured into two parts (see Figure 2). In the first part the interactive PSS are reviewed in detail, structured into three categories that are defined above: (a) interactive PSS applications, (b) interactive PSS tools, (c) participatory process. Category (a) looks at the context of the applications, i.e., the case studies, the sector, scale and the goals of the applications, category, category (b) studies the details of the hard- and software details of the interactive PSS tools and how they were developed, category (c) finally studies the characteristics of the participatory processes the interactive PSS were applied to including the methods of evaluation. The full list of review criteria, that have been defined based on a screening of all selected papers, is given below (Table 3).
The second part of the analysis aims at understanding how the studies covered in the review have addressed and measured impacts on stakeholder participation during the PSS workshops. In order to do so we analysed how the studies report on hypothesized or measured impacts on participation using the framework developed above (Section 3, Table 1). Statements of the papers regarding the impacts of the interactive PSS use on various aspects of participation were matched to the different evaluation criteria and classified into hypothesized or measured (see Appendix B, Table A4, Table A5, Table A6 and Table A7), depending on whether the authors aim to achieve the effects (i.e., hypothesized) or provide empirical evidence of having achieved them (i.e., measured). Where multiple statements in a study regarding the same dimension or criteria were found, these were counted as one in the final table (Table 4), because it is not relevant for the analysis how often a certain aspect of participation was mentioned. Negative statements indicating that certain dimensions of participation were not achieved were particular highlighted, because also these provide insights into the value of interactive PSS for facilitating participation. In cases where statements in the studies were not explicit but rather claimed general achieved impacts like social learning or better communication, these statements were classified in the appropriate dimension of participation.
The extraction of statements from the papers and their categorization into the different aspects and criteria of the framework was conducted by the first author. In order to minimize potential misinterpretation or omissions and to increase the reliability of such an intersubjective endeavour [14] the second author reviewed the results independently and added eventual missing statements. Disagreement between the two authors regarding the categorization of certain statements into the framework were resolved in a discussion of the entire team of authors.

5. Review Results

The presentation of the results is divided into two parts. First, the details of the applications and cases studies, tools developed and participatory processes they were used in are presented. Second, we analyse how and what the studies reported on the impacts on participation, based on the framework developed in Section 3.

5.1. Interactive PSS Applications, Tools, and Participatory Processes

The development of interactive PSS included in this review ranges from 2006 to 2018. Nonetheless, half of the applications have been developed and applied in PSS workshops between 2016 and 2018 (see Table 2). This shows that the use of such application has become more widespread and common in recent years. Detailed results of the review of the various interactive PSS are given in Appendix A, Table A1, Table A2 and Table A3.
The applications taken into account in this review cover a wide range of topics within the domain of spatial planning, from rural land use planning to urban redevelopment at neighbourhood level (Appendix A, Table A1). Next to common topics such as sustainable urban development, the sectors of urban health (3 applications), energy transition (4 applications) and climate change (3 applications) are predominantly addressed, which reflects typical foci of contemporary planning debates of the last 10 years. Half of the applications include at least one Dutch case study, which has to do with the fact that various research groups in the Netherlands work on the topic of interactive PSS. The goals of the applications are either related to designing spatial interventions such as adaptation measures, or to analysing the current situation and developing e.g., a good understanding of the problem and exchanging knowledge about it. Hardly any study covers the final phase of a decision-making process in which typically one option among a set of alternative designs is chosen, based on a proper evaluation.
All interactive PSS tools make use of a similar hardware setup of a maptable most often as a stand-alone system, plus in some cases an additional second screen e.g., for showing results of spatial interventions on the table in the forms of charts (Appendix A, Table A2). Maptables are operated either with special stylus pens or with fingers, only one study (ID 8) explores the use of physical objects (tags) to interact with the maptable. One PSS tool (ID 5) used a maptable that allowed distinction of inputs from different users. However, this was obviously not relevant for the results, because it was not made use of in the analysis of the workshop, nor has it prevailed in later studies.
The use of software for the tools reveals the fact that so far, hardly any software particularly made for maptables is available, with the exception of Phoenix, a participatory mapping tool [67], used in study ID 11. Most of the tools, therefore, use various ESRI ArcGIS components or an extension for the same, often CommunityViz Scenario 360 [68], which increases possibilities of dynamic user interaction with the geospatial content. Only one of the 16 tools considered here is developed based on an open source platform.
The GIS functionality mainly used by stakeholders during the interactive sessions comprises both functions for sketching and drawing features, as well as functions for calculating outcomes of certain interventions in the form of indicators, often visualized in charts. Since the limited capacity of the used GIS platforms for advanced modelling, some tools couple the GIS with a dynamic simulation model, such as a hydrologic model for calculating outcomes of interventions (ID1, ID 4b). Only three tools make use of 3D visualization for presenting spatial data. Also the inclusion of stakeholders in the development of the tools, as recommended by several scholars [13,15] in order to increase the usefulness of the tools, is hardly practised up to now, with the exception of studies IDs 4, 9, 13 (Appendix A, Table A2).
The planning phase most often addressed in the participatory processes supported by the tools is the design phase (Appendix A, Table A3). In the design phase, different stakeholders develop together measures or interventions in order to address given problems or planning challenges. In more recent applications, the problem analysis phase is also aimed at, in which the interactive PSS typically supports in gaining a thorough understanding of the given problem or an integration of knowledge prior to the design of interventions. Almost all processes involved expert stakeholders, either academics or practitioners from different domains. Only four processes (IDs 5, 7, 9, 13) aim at including also layperson, i.e., ordinary citizens, in the participatory processes.
In all processes, the participants are guided through more or less clearly defined steps or assignments that build up upon each other. Six studies combine the use of the maptable with other participatory activities such as rich picture drawing (ID 11) or GPS tracking (ID 12). The more concrete the steps are that the stakeholders have to go through, the less seems to be a need for extensive moderation. In tool ID 5 e.g., the participants first had to design suitable bicycle paths and then to evaluate land use scenarios [10]. On the other hand, for rather open processes a moderator has a relevant function to guide the process and engage the stakeholders. The use of a technical facilitator or chauffeur, who helps in handling the PSS, is common in recent participatory processes.
The early studies are not very explicit about the methods applied for evaluating the results and outcomes of the workshops or do not discuss these at all. Later studies applied a variety of methods for evaluating workshop processes and outcomes. Most common are post workshop questionnaires in which participants assessed various factors of usability of the tool as well as usefulness of the workshop process and outcomes. Seven studies combined various methods for evaluation in a sort of triangulation in order to answer various research questions or to crosscheck plausibility of findings. Only two studies applied questionnaires prior to and after the workshop in order to determine behavioural changes or learning outcomes (IDs 7 and 15).

5.2. Dimensions and Aspects of Participation Reported in the Studies

In this section, we analyse how and what the studies reported as impacts of the interactive PSS implementation on participation. The overall results are presented in Table 4 (for details see Appendix B, Table A4, Table A5, Table A6 and Table A7). The table shows how many of the in total 16 studies have explicitly reported statements related to the dimensions and criteria of participation as elaborated in the framework (Section 3, Table 1). Hypothesized and measured impacts are separated. Measured impacts are distinguished in positive and negative claims, the latter indicating aspects of participation that have not been achieved. Multiple claims that are collected for the same criteria or dimension are counted as one impact, as explained above (Section 4). Claims that could not be related to specific criteria of participation were associated with the relevant dimension of participation as an unspecific impact.
Table 4 shows that for some dimensions a huge difference between hypothesized and measured criteria of participation exists. In many studies, authors measured more detailed impacts on participation than they hypothesized upfront. In fact, several studies lack a proper framework for evaluating impacts on participation. While in some studies this was according to the authors also not the aim of the study (IDs 1, 2, 3), other studies failed to develop upfront a clear framework. Leao et al. [22] e.g., claim to test the performance and usefulness of the tool, but their study lacks an operationalization towards this goal. In other studies, the applied framework for evaluation towards participation impacts is introduced rather implicitly while explaining the questionnaire for the evaluation [61]. While the study ID 16 [65] has not stated an explicit focus on strengthening participation by using the PSS, some implicit references towards supporting participation are given, e.g., strengthening communication. On the other hand, another group of studies (IDs 6, 11, 14) developed upfront a dedicated framework for evaluating impacts on participation, which is often focusing on a specific dimension such as social learning.
Particularly in the process dimension of participation, a number of studies is making no distinction between the different criteria of participation. Both hypothesized and measured impacts are reported to support communication and collaboration (e.g., [9,64]) or trigger learning (e.g., [11,62]), without distinguishing what aspects of communication and collaboration or learning are addressed. In contrast to that, in the outcome dimension of participation impacts are more clearly distinguished both, while being hypothesized or measured. Different outcomes of a PSS workshops such as achieving a consensus is obviously much more concrete to grasp the rather vague notions of learning [69].
In total, the reviewed studies have mainly aimed at communication and collaboration as the process dimension and issue related outcomes as the outcome related dimension (Table 4). Those studies that have investigated communication and collaboration in more detail hypothesize mainly fairness (4 studies) and efficiency (3 studies) being the main goals of the interactive PSS workshops and applications. Interestingly, the criteria of inclusiveness and efficiency of the process are more often measured than hypothesized upfront. This supports the findings of the limited structured evaluation frameworks developed prior to the application. Learning during PSS workshops as an impact of participation is taken into account explicitly only in three newer studies (IDs 6, 11 14). These studies have hypothesised to investigate impacts of interactive PSS workshops on social learning and have also measured these.
Issue related outcomes are the by far most often hypothesized as well as measured dimension of participation. Nine studies aim at creating or integrating knowledge by means of interactive PSS workshop, while only six studies measure these. Three respectively four studies aim at achieving a consensus and/or increasing the problem understanding, but in six resp. 8 studies authors have measured these impacts. Interestingly, the criteria of improving the quality of the decision is only hypothesized in two studies, but measured as an issue related outcome in seven studies, four of them finding mainly positive indications, and three studies also reporting negative statements.
Finally, social outcomes are hardly addressed as hypothesized criteria in the studies. Increased ownership and commitment are hypothesized only in one study and both mutual understanding and satisfaction are only hypothesized in two studies. This shows that authors do not expect much on that dimension or do not consider that to be relevant in a PSS workshop, probably because it is often a one-time activity. On the other hand, in particular an increased mutual understanding among participants of a workshop is measured as a social outcome in nine of the studies, showing that such workshops in fact do have an effect on social aspects, at least at short term, i.e., at the time when the evaluation was done. With the same argument the creation of ownership or community building might not be considered, because these kind of impacts in fact require much more frequent and an intense working on an issue over a longer duration.

6. Discussion

6.1. Interactive PSS Applications, Tools and Processes

The interactive PSS analysed in this review reveal a large variety of applications, goals, contexts and case studies for different planning topics, illustrating the broad applicability of such applications. The more recent applications have a clear focus on issues at an urban scale [9,12,21,51,64], which on the one hand might reflect the increasing importance of urban topics in international planning debates [70]. On the other hand, this might also be due to the fact, that more small-scaled data is available nowadays enriching such urban studies and applications. Different case studies have been applied in different stages of a planning process, illustrating the potential value of these in various phases of planning. Some studies even tested the application of interactive PSS throughout different phases of a planning process [10,60], thereby demonstrating the value of such tools for supporting stakeholder participation in planning.
Nevertheless, most of the interactive PSS studies are developed and implemented in a research context, while only a minority of studies (IDs 6, 13) have been initiated through a municipality or planning agency. In the case of Pelzer et al. [11], the municipality “brought together an urban design firm, several environmental research agencies and a geo-communication firm to concretize its vision of the new neighbourhood” [11] (p. 177). The stakeholder workshops using the interactive PSS were perceived useful by all participants, but “the commercial developers (who owned most of the land) […] had not been involved in the MapTable sessions […] which led to a pause in the project execution.” [11] (p. 178). Moreover, no study discusses how the results from the PSS workshops are further used in the planning process. In the study of Flacke and de Boer [9] “the workshop results […] were summarized in a policy “document [71]. However, no follow up steps have been taken so far” [9] (p. 14). One can conclude that while the tools and applications presented in the studies might have the potential to overcome the widely discussed PSS implementation gap [6,72] any proof for that is still lacking.
The analysis of PSS tools shows that different types of hardware setups and infrastructure systems are being implemented successfully. In contrast, there is an obvious demand for the development of dedicated software tools supporting a group process on maptables. In most cases, generic and off-the-shelf GIS software augmented with certain extensions or extra functions is being used, which is not designed for tangible user interfaces, resulting in limited user friendliness of the tools. No open source tools such as QGIS, holding the promise of permanent refinement and tailor-made modification through a large user community [73], are being used. Moreover, only one study (ID 11, [12]) uses a tool that is explicitly developed for tangible user interfaces, but that tool supports basically participatory mapping studies lacking analytical functions of a PSS. In conjunction with the findings that the tool development processes hardly involve any stakeholders [14,15], we speculate that the development and implementation of such maptable based PSSs are still the job of experts and consequently the tools are far from being used in every day planning practice.
Finally, only very few tools reviewed make use of 3D visualization techniques, though this is reported to enhance participation, particularly of layperson, in planning significantly [74]. In line with that, PSS workshops reviewed here are mainly conducted involving planning experts as stakeholders, either researchers or planning practitioners, who are likely to be experienced in working with maps and 2D visualizations of spatial data [8]. Laypersons, such as ordinary citizens, are involved only in 4 of the 16 studies. From our own experience [9] this might be explained by the non-availability of user-friendly PSS tools and interfaces, e.g., tools made explicitly for tangible user interfaces, which would be easier to use also for non-experts. Vonk et al. [75] found in a study on the bottlenecks in the uptake of PSS that using such tools […] depends, among other things, upon previous experiences in planning practice. The vast potential offered by user friendly maptable applications and attractive (3D) visualization for involving particular non experts is hardly explored yet [8].

6.2. Interactive PSS Contribution to Participation

This review shows that all interactive PSS applications and tools succeeded to facilitate intensive, in some cases, enthusiastic [37] stakeholder collaboration and workshops participants reacted predominantly positive to the possibilities provided by the PSS tools. Thus, we can conclude that in all cases studies the interactive PSS has contributed to strengthening participation, as it can be seen from the results in Table 4. However, the most apparent finding is that many interactive PSS case studies do not make use of a very distinct and explicit framework in order to evaluate impacts of the interactive PSS tool on participation. Thus, impacts on participation are often being assessed rather incidentally based on implicit assumptions [15], without being underpinned by a proper conceptual or theoretical discussion. Having said this, we can conclude that many of the studies reviewed here can be characterized as studies testing the usability of the tools rather than their usefulness for the planning practice [14]. However, given the obvious strength of many of the interactive PSS for improving aspects of participation, detailed frameworks of usefulness or performance of PSS as initiated in contemporary scholarly debates [8,14,15], are needed in order to equip practical and applied studies with a proper basis and to make studies better comparable.
Given the different orientations of reviewed studies regarding the goals of participation, foci on different aspects vary accordingly. However, the focus of participation in general is mainly on how interactive PSS help facilitating the communication and collaboration and on issues related outcomes that can be achieved. While a few newer studies focus explicitly on the impacts of interactive PSS on aspects of learning [11,12,51], no studies have yet focused on analysing the social outcomes of PSS workshops. This matches with the observation of Goodspeed and Pelzer [34] that it is more likely that process attributes are measured, because outcomes are often hard to see, because they develop over a longer time period. Nevertheless, given the increasing need to engage with stakeholders under current planning regimes [76] to engage with stakeholders for the co-production of services [77], it is interesting for future studies to investigate what kind of social outcomes can be supported by means of interactive PSS applications. Therefore, PSS tools would have to be applied over a longer period, e.g., a series of workshops.
Last main finding worth to discuss here is that obviously scholars tend to report what works well, rather than what does not work. In all papers related to the different studies in total 176 positive quotes (including multiple quotes in one paper referring to the same criteria of participation) were found relating to one of the different aspects and criteria of participation, and only 20 negative statements (see Appendix B). It seems that authors are predominantly looking for evidences that prove their hypotheses and thereby devaluing negative findings, as it is rather unlikely that in only a bit more than 10 % of all instances, they did not succeed in doing so. This phenomenon known as the publication bias is reported by other scholars and occurring in various disciplines [78,79]. Fanelli [80] (p. 892) observes “various forms of conscious and unconscious biases that affect all stages of research […] producing positive findings when there should be none, thus creating distortions that are difficult to correct a posteriori”, particular “in fields where theories and methods are less clearly defined, and true replication is rare or impossible” Having tested 4600 papers from different disciplines over a period of 17 years (1990–2007) he further hypothesizes that research is becoming less pioneering and that the objectivity with which results are published is decreasing [80]. Nevertheless, as the exact number of positive or negative statements is not relevant for this study, because multiple quotes are counted as one per study (see Section 4), this observation does not significantly affect the reliability and relevance of our findings.

6.3. Reflection of the Methodology Applied in this Review

The methodology applied in this review includes various aspects that might limit the validity of the obtained results. First, the characteristics of the various interactive PSS presented in Appendix A, Table A1, Table A2 and Table A3 (applications, tools, processes) result from the interpretation of the authors if explicit details were not given. Moreover, when a ‘no’ is given in the tables it means it was not given in the paper, which does not necessarily mean that it was not done. It can of course be that the authors of the study simply did not explain it. Second, the hypothesized and measured impacts of the various studies on the different aspects and criteria of participation as analysed in Section 5.2 (see also Appendix B, Table A4, Table A5, Table A6 and Table A7) is an intersubjective endeavour, particularly when hypotheses are not explicitly spelled out by the authors. In order to minimize this subjectivity and to increase inter-rater reliability [81] all statements were cross-checked by the co-authors. Third, certain impacts of the use of interactive PSS in stakeholder workshops that were reported by the authors were not taken into account, due to the theory driven analysis method applied [16]. For instance, some studies reported that the use of the maptables created enthusiasm among participants [20,37]. Enthusiasm is for sure something the use of a maptable might create, but in our view, it does not necessarily lead to better or more participation. Therefore, it is also not included in any of the studies used here to develop the framework in Section 3. On the contrary, in our own studies we have observed enthusiasm rather limiting a serious and productive use of the interactive PSs, and therefore would have to be overcome in order to institutionalize such tools in everyday practice.

6.4. Future Research Needs

Findings from the review allow identifying the following future research needs in relation to interactive PSS. First, most tools reviewed here are developed using proprietary GIS software which is in most cases not explicitly developed for tangible user interfaces. Open source tools offering potentials to further refine applications by others or in future case studies are so far very much neglected. Future research studies should explore impacts of interactive PSS tools made explicitly for tangible user interfaces on their user friendliness and its correlation with the usefulness of the tool [14,82].
Second, only a minority of reviewed studies reported the inclusion of users and/or stakeholders in the PSS tool development process. A stronger integration of them in the development process is recommended by scholars as being integral for enhancing uptake and implementation of PSS [15,83]. Thus, future interactive PSS studies shall research the challenges as well as added values of including users and stakeholders in the tool development process, e.g., following a human centred design (HCD) approach [84].
Third, the review revealed that social outcomes of the use of interactive PSS in stakeholder workshops are hardly researched yet. Thus, in suitable long-term case studies potential social outcomes such as community building or increased ownership through interactive PSS workshops series might be investigated. In this context te Brömmelstroet [82] suggests to research how more flexible PSS that that can be quickly adapted and contextualized [85] allows the actors to develop a sense of ownership and adapt the PSS to their specific demands.
Finally, the adoption of results from interactive PSS workshops into ongoing planning processes needs to researched. To this end Geertman [6] recommends to study successful or best practice studies and to learn from them. Te Brömmelstroet ([83] p. 80) proposes a research programme that combines “context-rich and control-rich [PSS] research environments” in order to examine the uptake of PSS research in planning practice.

7. Conclusions

This review analysed how interactive PSS have contributed to the goal of strengthening stakeholder participation. Findings from the 16 studies reviewed here show that the implementation of maptable-based interactive PSS in planning related studies has increased significantly since the first case study in 2006 in numbers of tools developed, geographic regions covered, and in the variety of topics and tasks addressed. Over the years, not only the tools and applications have become more sophisticated and complex, also the goals of the studies using such tools have changed from involving stakeholders in designing interventions [20,55] to involving stakeholders in earlier planning stages, e.g., in order to develop a joint problem understanding or integrating knowledge [12,51]. In line with that, the interest of researchers in such studies has evolved from basically exploring the usability of interactive PSS for stakeholder participation [57] to more explicit research questions related to stakeholder participation such as the added value of interactive PSS on group-based learning [37,55].
Based on the second part of the analysis we can conclude that in all case studies the interactive PSS has contributed to strengthening participation, because all studies reported positive statements regarding the different aspect sand criteria of participation. Nevertheless, a key finding is that many case studies of interactive PSS reviewed here lack a proper framework and operationalization for investigating the effects of the tools and applications on the different aspects and criteria of participation. Though studies usually hypothesize certain impacts of their tools on participation and apply suitable methods for measuring and evaluating such impacts in PSS workshops, in many studies a certain mismatch was detected. Many studies are much more detailed in measuring the impacts on participation, e.g., by evidencing impacts on the transparency or inclusiveness of the public dialogue, than hypothesizing them before. Other studies provide detailed hypotheses of impacts but fall short in evidencing them by means of evaluation. Only three of the reviewed 16 studies, come up with a proper framework upfront and systematically measure impacts of those aspects of participation hypothesized upfront. In conclusion, further theoretical studies conceptualizing impacts of interactive PSS on participation are needed as well as more empirical studies, testing these impacts in real world case contexts with various groups of stakeholders.
Moreover, most studies analysed result from a more or less strict research context. Consequently, the adoption of the tools and applications in planning practice, e.g., through the involved case study partners, is not reported. Thus, no straight conclusions can be drawn from the review regarding whether or not the implementation of interactive PSS might have the potential to mitigate the often-stated PSS implementation gap. On the contrary, findings from the review help identifying issues that needs to be addressed in future studies in order to mitigate the implementation gap, such as the use of open source software explicitly made for maptables, the inclusion of users in the PSS development process, and the in depth analysis of results from PSS workshops in planning processes (see Section 6.4).

Author Contributions

Conceptualization, Johannes Flacke; Methodology, Johannes Flacke, Rehana Shrestha; Validation, Johannes Flacke, Rehana Shrestha; Rosa Aguilar; Formal Analysis, Johannes Flacke, Rehana Shrestha; Rosa Aguilar; Data Curation, Johannes Flacke; Writing—Original Draft, Johannes Flacke; Writing—Review & Editing, Rehana Shrestha; Rosa Aguilar; Visualization, Johannes Flacke. All authors have read and agreed to the published version of the manuscript.


This research received no external funding.


The authors thank Karin Pfeffer for providing valuable comments on a draft version of the paper. We also thank the anonyms reviewers for providing useful comments that helped to improve the paper.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A. Analysis Part One—Details of Reviewed Interactive PSS

Table A1. Interactive PSS applications.
Table A1. Interactive PSS applications.
IDCase StudiesSector/Field of ApplicationScaleGoal of the Application
1Ijssel river clos to Deventer, NetherlandsWater managementRegionalDesigning spatial measures for controlling water levels of the river
2Groningen province, NetherlandsClimate change adaptationRegionalIntegrating sectoral adaptation measures
3South West Victoria, AustraliaForest regeneration planningRegionalDesigning biolinks (ecological corridors) for forest regeneration
4Three case studies:
(a) Stedendriehoek area,
(b) hypothetical area,
(c) Brabantse Delta (all areas in the Netherlands)
Water related land use planning (a, c), energy transition (b)RegionalExploring available information, exchanging ideas about the current and future situation, sketching potential interventions
5Two case studies:
(a) Bodegraven polder, Netherlands,
(b) Mull of Kintyre, Scotland
(a) Rural land use planning,
(b) offshore renewable energy planning
RegionalInvolving stakeholders in entire planning process of analysis of the situation, design of potential interventions, choice of agreed upon interventions
6Four case studies:
(a) Rijnenburg, Utrecht,
(b) Arnhem,
(c) Deventer,
(d) Achterhoek (all areas in the Netherlands)
(a) Sustainable urban development,
(b) energy transition,
(c) rban design,
(d) adjustment of urban development plans
Urban, city to neighbourhood levelFacilitating communication among stakeholders during different planning stages
7Three case studies:
(a) Hommerts,
(b) Veenpolder,
(c) Buitenveld (all areas in the Netherlands)
Rural land use planningRegionalDesigning a spatial adaptation strategy for land use related water management in a peat meadow area
8Two case studies, each conducted in multiple areas:
(a) Esch-sur-Alzette, Luxembourg; Ludwigsburg, Germany,
(b) London, UK; Brussels, Belgium; Luxembourg
(a) Climate change mitigation,
(b) urban logistics
UrbanTranslating policy visions into concrete projects
9Six case studies: Beira, Mozambique; London, UK; Oaxaca, Mexico; Utrecht, Dordrecht, Tilburg, all NetherlandsClimate change adaptationUrbanDefining green and blue adaptation measures
10Liemers Corridor, NetherlandsRegional development planningRegionalNegotiating distribution of commercial and industrial development locations at regional level
11Dortmund, GermanyEnvironmental healthUrban, city to neighbourhood levelMapping stakeholder knowledge on environmental health issues
12North-East Scotland, ScotlandLandscape planningUrban and rural, regional scaleExploring spatial behaviour in the local landscapes
13Enschede, NetherlandsEnergy transitionCityDiscussing locations of renewable energy projects
14Two case studies: Dortmund and Munich, GermanyEnvironmental healthCity districtassessing and mapping multiple environmental burdens
15Sydney, AustraliaUrban redevelopmentUrban, precinct levelDeveloping different scenarios of urban redevelopment
16City of Hume, Melbourne, AustraliaHealth related urban planningUrban, city levelexploring built Environment scenarios regarding impacts on walkability and designing interventions
Table A2. Interactive PSS tools.
Table A2. Interactive PSS tools.
IDHardware Setup/System ArchitectureSoftwareMain GIS FunctionsExternal Model CoupledStakeholders Involved
1Maptable used with stylus pens, stand alone systemArcGIS plus interface panel for maptable (Woosh)Drawing features, sketching, annotating, queries, calculation of effectsHydrologic model (Waqua), spatial schematization tool (Baseline)No
2Maptable, stand alone systemArcGISDrawing, visualisationNoNo
3Maptable, stand alone systemArcGIS plus landscape constructor extension and visualisation tool (SIEVE)Drawing polygons, assigning vegetation types, 3D visualizationLandscape object libraryNo
4Maptable, clinet-server structureArcObjects technology, various librariesSketching, indicator calculation4b) Energy performance modelLab based study beforehand, (see [25])
5Maptable (Diamond Touch™) 3, stand alone systemCommunityViz Scenario 360 1, DT collaborate (ArcGIS plugin)Drawing, MCA, allocationNoNo
6Mapsup MapTable®, stand alone systemCommunityViz Scenario 360 1Drawing, indicator calculation, visualisationSustainability Profile of the Location (SPL) score calculated based on indicatorsNo
7Maptable, SUR 40 Samsung, stand alone systemCommunityViz Scenario 360 1Geodesign tools, indicator calculation,NoNo
8Maptable (GTUI) used with physical objects (tags), web-basedPostgreSQL1 server with geospatial extension; client: Java application (open source)Place tagged objects on the screen for calling certain functions such as split screen, data layersNoNo
9Maptable, stand alone systemNot givenAllocate predefined interventions, ranking of interventions, performance indicator calculationNoTest cases of the tool run in Rotterdam and Delft (see [21])
10MapsUp MapTable®, stand alone systemCommunityViz Scenario 3601(Re)-allocating pre-configured building blocks of development space, indicator calculationsNoNo
11Maptable plus extra screen, stand alone systemPhoenix 2Sketching, annotating with symbols 3D visualizationNoNo
12Maptable, Samsung SUR40 with Microsoft Pixelsense, web-basedBing mapsPan and zoom, no particular GIS functions usedNoNo
13Maptable plus extra screen, stand alone systemCommunityViz Scenario 360 1Sketching, indicator calculationsNoDiscussions of the model with the municipality
14Maptable plus extra screen, stand alone systemCommunityViz Scenario 360 1Sketching, multi criteria analysis (MCA)NoNo
15Maptable, stand alone systemArcGIS toolboxInput of model parameters, scenario calculation, 3D visualizationNoNo
16Maptable, stand alone systemCommunityViz Scenario 360 1Sketching, changing model assumptions, indicator calculationCustom regression models built into CommunityVizNo
1 ArcGIS Extension [68], 2 by Geodan [67]. 3 This maptable system allows user recognition.
Table A3. Participatory process.
Table A3. Participatory process.
IDPlanning PhaseStakeholders/ParticipantsPSS-Supported StepsOther Participatory StepsProcess FacilitationProcess Evaluation
1DesignPractitioners from city, ministry, forestry, infrastructure agency, architectThree stages: define measures, calculate effects, evaluate resultsEvaluation of design done verbally based on calculated effectsProcess and technical facilitatorNot given
2ChoiceDifferent sectoral expertsThree stages: map sectoral measures, integrate spatial measures into one coherent map, agree on integration mapMeasures first sketched on paper mapsModerator to operate the GIS on the maptableNot given
3DesignLandscape planners, scientistsDesign and then visualisation in 3DNoNot given5 point likert scale questionnaire on usability of tool
4Problem analysis and designPractitioners (landscape architects, planners) and academicsThree stages: area exploration, selection of suitable locations, design of new locationsNoNoObservers (development team and experts) filling evaluation questionnaires, discussion of findings with participants; stated experiences of participants
5Problem analysis, design, and choiceDesign workshop: 10 stakeholders (municipality, waterboard, NGO, etc.); choice workshop: local stakeholders, citizens, and municipalityTwo workshops, local knowledge and negotiation workshop, each workshop having 3 sessions: intro to research problem, familiarise with tools, participants use of tools to fulfil the objectivesDesign on paper maps in parallel groups, use of DEFINTE MCA in design workshopnoSurvey at the end of each workshop, plenary discussion
6Problem analysis, design, and choice; partly addressed in separate case studiesDifferent practitioners: environmentalist analysts (noise, air quality, water management), urban designersVarious number of workshops in different cases workshops;NoTechnical facilitation on using the PSSSurvey questionnaire in negotiations workshop
7Problem analysis and designFarmers, interest groups, NGOsThree assignment in the workshops, related to three strategiesNoNoSurvey done before and after the workshop
8DesignLocal stakeholders: urban planners, GIS technicians, social scientists, engineers, energy experts, and companies; municipalities, public transportationTranslate vision into concrete projects); understanding the contextNoNoQuestionnaire
9DesignVarious local experts: municipal staff members, planners, local experts, NGOs, partly also citizensFirst maps and 3D visualization were shownLoosely linked to climate adaptation app [86]Not givenQuestionnaire
10Not strictly related to a typical planning processregional urban development practitioners from province and municipalitiesTwo workshops: various rounds of playing with different tasks, fociNoNoVideo recording, three questionnaires during the workshop regarding expectations
11Problem analysisPractitioners from planning and health (municipality, agencies, NGOs)Three phases: explore existing information, add tacit knowledge, integrate knowledgeRich picture drawingFacilitator for substantial and procedural issuesobservations, questionnaire, oral feedback, screen and voice recording
12Not given22 participants, individuals, most of them being researcher on land based issuesJoint discussion self-recorded GPS walking routes visualised on the maptableCombined with smart phones GPS tracking of walking routesNoIndividual interviews 4-6 weeks after workshop, video recording of the sessions
13Problem analysisCitizens, policy makersTwo stages, focussing on different types of renewable energyNoModerator and chauffeurQuestionnaire, screen and voice recording
14Problem analysisResearchers and practitioners (municipality, housing association, NGO)Part of a one day planning gamePlanning game on identifying economically and socially deprived districts, and communitiesModerator and chauffeurQuestionnaire, observations, screen and voice recording
15Problem analysis and designUrban plannersSession structured into five steps related to five different questionsNoTechnical facilitatorPre and post workshop questionnaire
16DesignUrban planners and one health expertSketching interventions after intro to the tool, compare scenariosNoTechnical facilitatorQuestionnaire, group discussion

Appendix B. Analysis Part Two—Reported Impacts on Aspects and Criteria of Participation

Table A4. Communication and Collaboration.
Table A4. Communication and Collaboration.
H = 2H = 4H = 2H = 3H = 12
M = 1M = 2M = 7M = 4M = 12
M = 1M = 2M = 2M = 2M = 3
1 [20]“The design [of the maptable interface] should address [...] efficiency [...] properly.” (p. 265)efH
“... Very suitable in brainstorming, quick and easy to present ideas, empowers collaboration (p. 270)CM
“Designing is equally possible for all participants.” (p. 270)inM
2 [55]“...the table setting was expected to better facilitate face-to-face communication...” (p. 7)CH
“...the [...] table was expected to entice participants to interact and share on the table.” (p. 7)CH
The table setting allowed four to five people at a time to view the work and communicate comfortably. (p. 7)CM
“The simultaneous projection of intermediate steps and resulting maps on a wall enabled larger groups to follow the work in progress...” (p. 7)inM
“This led to a moderate degree of group participation.” (p. 7)CM
3a [57]“...such a suite of geo-visualisation tools will enable active participation and exploration of decision spaces by a number of end users.” (p. 331)CH
“enables and engages participants in an interactive session bringing together researcher, landscape planners and policy-makers in a participatory way” (p. 331)CH
“...using this approach helps the user to make changes in their 2D plan […] and view them interactively in the 3D environment immediately...” (p. 338)efH
3b [56]“All participants can view the designed biolink…”faH
“...interactive participatory decision making approach to involve multi-stakeholders in landscape.”CH
4a [58]“...enable experts and citizens to exchange ideas in an efficient and accessible way.” (p. 8)efH
“...invites participants to engage actively in a process of communication ...” (p. 8)CH
“The MapTable environment stimulated discussion...” (p. 13)CM
“...the user who is holding the pen dominates the session...” (p. 14)faM
“The use of the MapTable keeps all the participants active.” (p. 14)inM
“...leading to relevant and well-informed discussions.” (p. 16)CM
4b [59]“Each stakeholder [...] has the opportunity [...] to contribute its own ideas...”inH
5a [10]“...maps are used as the means of communication to facilitate stakeholder dialog”. (p. 334)CH
“Stakeholders are invited to work together [...] and share their views...” (p. 334)CH
“To communicate information about land use suitability” (p. 334)CH
“ were effective for communicating large amounts of information...” (p. 337)efM
“prompted input from all participants...” (p. 337)inM
5b [60]“During the ‘local-knowledge’ workshop all contributors participated enhancing interactivity and communication across sectors.” (p. 7)inM
6a [11]“...urban designers from the external firm were less actively involved...” (p. 178)inM
“...otherwise the momentum would have been lost.” (p. 179)efM
“…leading to sensitivity to technical errors and lengthy calculation times.” (p. 179)efM
“...of the environmental analysts found it difficult to think in terms of scores of 1–10...” (p. 180)inM
“...would be successful if the urban designers would stay around MapTable until the end...” (p. 180)inM
“...intensive discussions about the incorporation of environmental issues in the spatial plan...” (p. 180)CM
“...measures couldn’t be presented right away, people lost their concentration...” (p. 181)efM
“...leads to an open and constructive group dynamics...” (p. 182)faM
6b [37]“...the PSS was utilized to support this collaboration...”CH
“...the MapTable PSS clearly involved [...] communication support...”CM
“It enabled the different professionals to explore together the sustainability Consequences...”CM
“ had a broader aim of supporting a constructive dialogue with increased collaboration and communication...”CH
“...were enthusiastic about how this information on consumption patterns was visualized and communicated...”CM
“ its ability to keep participants active around it, their attentions focused on the information...”efM
“...the MapTable PSS [...] led to a fair, clear and transparent selection process...”fa, trM
“...participants could directly be addressed by other participants and convinced to participate in the collaboration process...”CM
7 [38]“...provided a common platform for discussion.” (p. 104)CH
“...did well in facilitating exchange of information...” (p. 113)CM
8 [61]“...Users should be able to talk to each other face-to-face...” (p. 189)CH
“...provide a new, common language for inter-discussion.” (p. 189)CH
“...each of them playing an equal part in the discussion.” (p. 190)faH
“...communicate differently [...] and will collaborate more...” (p. 196)CH
“...stakeholders to discuss longer...” (p. 196)efM
“...the discussion process will get shorter and that a solution will be found faster...” (p. 197)efM
“...allowing stakeholders to express themselves in a clear way... “(p 197)inM
“...allows everybody to participate...” (p. 198)inM
“...encourage communication...” (p. 198)inM
“...some citizens will have problems to understand everything.” (p 198)inM
“...helpful for brainstorming with other people or to convince a politician of a strategy...” (p. 201)CM
“...multiple participants actively interacted...” (p. 202)inM
9 [83]“...discuss alternative measures...” (p. 428)CH
“...facilitates reasoning together...” (p. 429)CH
“...effectively supported climate-proof planning...” (p. 433)efM
“...details of the plan were discussed among participants at the design table...” (p. 434)CM
10 [62]“...facilitated a focused form of communication...” (p. 84)CM
“All participants pointed to relevant features on the map...” (p. 84)inM
“ order for the negotiation to work, insight in the figures is essential...” (p. 85)trM
“...helps to structure the discussion...” (p. 85)efM
“...used the Maptable more as a discussion tool...” (p. 86)CM
11 [12]“ enable communication...” (p. 4)CH
“...listening to each other, treating each other’s interests with respect...” (p. 5)faH
“...discussing their own perspectives and knowledge...” (p. 10)CM
“...information and knowledge are communicated...” (p. 11)CM
“...tensions and resistances also surfaced...” (p. 11)CM
12 [63]We observed that interaction with the touch table can be dominated by one or two individuals...” (p. 13)faM
“... can make it easier for more reserved participants to contribute.” (p. 13)faM
13 [9]“...encourages stakeholders to discuss jointly...” (p. 3)CH
“...various groups focused [...] more on achieving the given goals rather than on an eventually conflicting discussion of potential locations...” (p. 10)CM
“...allows better comunication...” (p. 12)CM
14 [51]“ dialogue [...] and an exchange of each other’s perspectives...” (p. 5)CH
“...transparency in the process...” (p. 5)trH
“ active dialogue among the participants...” (p. 13)CM
“ of perspectives amongCM
Participants...” (p. 13)CM
“...Collaborative relationships were also evident...” (p. 17)
16a [64]“...two-hour participatory workshop to facilitate dialog [...] among researchers and practitioners...” (p. 131)CH
16b [65]“ of transparency.” (p. 157)trH
“...sketch and test ideas together...” (p. 158)CH
“...supporting communication...” (p.159)CH
“...effective tool to support communication...” (p. 160)CM
“ consultation with other stakeholders and public...” (p. 161)CM
“...unclear whether the operation of the Walkability PSS on the MapTable itself was essential to support participation and engagement...” (p. 164)CM
Communication and collaboration (C); transparent (tr); fair (fa); inclusive (in); efficient (ef); H = hypothesized claims; M = positive claims measured; M = negative claims measured.
Table A5. Social learning.
Table A5. Social learning.
H = 2H = 4H = 2H = 3
M = 1M = 2M = 7M = 4
M = 1M = 2M = 2M = 2
5a [10]“...helped increasing their awareness of new aspects of the region’s problems” (p. 337)awM
6a [11]“ clearly capable of improving the learning processes of actors...” (p. 181)LM
“...this interaction did not necessarily lead to learning processes...” (p. 181)LM
6b [37]“...analysts indicated to have learned a lot from each other...”LM
9 [83]“...should raise awareness...” (p. 428)awH
“...impacts and vulnerable spots were recognized by the participants...” (p. 433)slM
“...supported a learning process...” (p. 433)LM
10 [62]“...learning (individual and group learning) as the most important quality criterion...” (p. 77)LH
11 [12]“ enable [...] social learning...” (p. 4)LH
“...leads to enriched insights and social learning...” (p. 14)LM
12 [63]“…had become more aware of other people she encountered on her walk...” (p. 11)awM
13 [9]“ raise awareness and facilitate social learning of stakeholders...” (p. 3)awH
“...achieving the goal of social learning...” (p. 11)LM
“...learned by the participants in the sense of social learning...” (p. 11)LM
“...learned something about renewable energy during the workshops...” (p. 12)LM
14 [51]“...intended outcomes of knowledge co-production and social learning...” (p. 3)LH
“...detected mismatch between expected and observed outcomes is corrected by revisiting the assumptions...” (p. 6)dlH
“...mismatch between expected and observed outcomes is corrected...” (p. 6)slH
“...Critical questioning of underlying assumptions of the model...” (p. 13)dlM
“Evident instances of single-loop learning...” (p. 15)slM
“Instances of double-loop learning occurred...” (p. 16)dlM
16a [64]“...two-hour participatory workshop to facilitate [...] co-learning among researchers and practitioners...” (p. 131)LH
Social learning (L); awareness raising (aw); single loop learning (sl); double loop learning (dl); H = hypothesized claims; M = positive claims measured; M = negative claims measured.
Table A6. Issue related outcomes.
Table A6. Issue related outcomes.
H = 2H = 4H = 2H = 3
M = 1M = 2M = 7M = 4
M = 1M = 2M = 2M = 2
1 [20]“After an agreement has been reached on a common design...” (p. 267)coM
“In the discussion [...] ideas for changing in the design in the next cycle were indicated. (p. 269)qaM
2 [55]“The resulting, integrated map was accepted by all participants...” (p. 8)coM
3a [57]“…reason for bringing together these spatial data layers is to enhance participants’ understanding and thinking about the study area.” (p.335)unH
3b [56]“…this gives users the opportunity to consider these factors in their biolink delineation”knH
“improved biodiversity and habitat plans can be created between…”qaH
“a more informed way of understanding environmental processes”unH
4a [58]“...invites participants to engage actively in a process of [...] sharing of knowledge.” (p. 8)knH
“landowners and formal organizations [...] try to come to a consensus.” (p. 9)coH
“In an iterative working process, ‘hotspots’ were continuously evaluated, discussed and changed.” (p. 15)knM
5a [10]“To improve stakeholder understanding of the problem” (p. 334)unH
“exchange of local knowledge and stakeholder dialog” (p. 334)knH
“To use this information to improve the quality of the plans” (p. 335)qaH
“To negotiate a land use plan that is perceived as acceptable by all Participants” (p. 335)coH
“… increased the credibility of the information presented...” (p. 338)qaM
5b [60]“... aim was to gather spatially explicit data from local sea-users...” (p. 3)knH
“...aim [...] was to build a consensus on the best locations...” (p. 3)coH
“Conflict [...] emerged throughout the workshop process.” (p. 3)coM
“ stakeholder maps were generated {...] based on the knowledge of the [...] stakeholders...” (p. 4)knM
“Participants added a considerable amount of data ...” (p. 4)knM
Discussion [...] regarding the quality of the data used led to suggestions that the published tidal data were inaccurate.” (p. 5)qaM
Results [...] indicated that 80% of participants found the combination of knowledge from researchers and stakeholders to be most helpful” (p. 7)knM
“...contributed to a broader understanding of the complexity of problems...” (p. 7)unM
6a [11]“...the structuration of local knowledge...” (p. 174)knH
“ helped them to envision the diverse topics of a planning problem in an interconnected way...” (p. 180)unM
“...If you emphasize integrality too much, it hampers focus and creativity.” (p. 180)qaM
“...lead to group dynamics in which it is easier to come up with ideas...” (p. 180)knM
“...understanding of the issues of relevance to other disciplines and the interrelatedness of problems...” (p. 180)unM
6b [37]“...which both provided insight in the planning issue...”unM
“...gained more insight into the plans of other municipalities and into the future economic consequences of these plans...”unM
“...the increased transparency through [...] the PSS did not always support reaching agreements...”)coM
7 [38]“...increased understanding of these relations...” (p. 113)unM
8 [61]“...better understand the maps...” (p. 196)unM type of overview [...], which is more understandable...”unM
“Stakeholders will discuss longer before reaching a consensus.” (p. 196)coH
9 [83]“...supports them in how to share their knowledge...” (p. 428)knH
“Two alternative plans emerged from this discussion...” (p. 433)qaM
“...forced participants to be explicit about their proposed interventions...” (p. 434)qaM
“ find a common preference with all participants...” (p. 434)coM
10 [62]“...used in-depth knowledge...” (p. 84)knM
“...identify and discuss locations [...] that otherwise might be overlooked.” (p. 86)qaM
“...successfully brings information together in interactive fashion...” (p. 87)knM
“...Learning about the issue...” (p. 87)unM
11 [12]“ develop a shared understanding about the problem situation...” (p. 4)unH
“...enable integration of explicit knowledge and stakeholders’ tacit knowledge...” (p. 4)knH
“...willing to put their “half baked” ideas forward...” (p. 5)knH
“...knowledge is shared easily...” (p. 11)knM
“...externalize and capture tacit knowledge...” (p. 12)knM
“...Consensus was reached...” (p. 14)coM
12 [63]“...method to facilitate discussions about people’s perceptions and knowledge of local landscapes.” (p. 1)knH
“…the touch table facilitated participants in gaining an understanding...” (p. 7)unM
“...the session around the touch table had helped to change the understanding [...] people had...” (p. 11)unM
13 [9]“...discussions towards consensual options and locations...” (p. 3)coH
“...high level of agreement between the participants...” (p. 11)coM
“...mapping results were perceived as less valuable and reliable...” (p. 13)qaM
14 [51]“...engaged in cooperative endeavors of knowledge production...” (p. 3)knH
“...participants may also come to understand...” (p. 4)unH
“...elicitation of stakeholders’ tacit knowledge...” (p. 5)knH
“...participants’ suggestions and knowledge were also considered.” (p. 12)knM
“ of knowledge among the groups...” (p. 16)knM
“...participants discovered common linkages between various interests...” (p. 17)coM
“...were able to agree on a set of indicators/indices...” (p. 17)coM
“...lead to collective action by the group...” (p. 17)coM
15 [22]“...combination of models in the tool facilitates the understanding of the role of different variables...” (p. 1389)unM
16a [64]enabled a group of spatial planners to [...] understand how this increased the probability of transport walkingunM
Knowledge created/integrated (kn); consensus achieved (co); understanding of the problem improved (un); quality of decision improved (qa); H = hypothesized claims; M = positive claims measured; M = negative claims measured.
Table A7. Social outcomes.
Table A7. Social outcomes.
H = 2H = 4H = 2H = 3
M = 1M = 2M = 7M = 4
M = 1M = 2M = 2M = 2
1 [20]The design should address [...] satisfaction properly (p. 265)saH
5a [10]“the Touch table helped increasing their awareness of [...] different standpoints as well as scopes of other stakeholders (p. 337)muM
6a [11]“...lead to group dynamics in which it is easier [...] to think beyond professional roles...” (p. 180)muM
6b [37]“...the PSS gets people off their islands, they come closer....”muM
7 [38]“...raised awareness of the perspectives of other Participants...” (p. 113)muM
9 [83]“...satisfaction with the way the planning process was structured...” (p. 433)saM
“...stimulates thinking beyond the own professional roles...” (p. 434)muM
10 [62]“...tool effectively brings motives to the surface...” (p. 87)muM
“...insight in each other’s current situation and future plans...” (p. 87)muM
11 [12]“...participants identify commonalities in views...” (p. 7)muH
“ of shared meanings was evidenced...” (p. 14)muM
12 [63]“ what the other person said made them remember other aspects of their walk...” (p. 11)muM
13 [9]“ increase community engagement...” (p. 3)owH
“ get to know the views of others...” (p. 11)muM
“...their views on renewable energy with the other participants...” (p. 12)muM
14 [51]“...shared understanding of the issues under investigation...” (p. 5)muH
“...insights into normative aspects and each other’s assumptions...” (p. 5)muH
“...participants were able to work through their interests linking these with others...” (p. 17)muM
Ownership/commitment (ow); mutual understanding (mu); community building (cb); satisfaction (sa); H = hypothesized claims; M = positive claims measured; M = negative claims measured.


  1. Healey, P. Planning through Debate: The Communicative Turn in Planning Theory. Town Plan. Rev. 1992, 63, 143–162. [Google Scholar] [CrossRef]
  2. Healey, P. The Communicative Turn in Planning Theory and its Implications for Spatial Strategy Formation. Environ. Plan. B Plan. Des. 1996, 23, 217–234. [Google Scholar] [CrossRef]
  3. Friedmann, J. Thinking about complexity and planning. Int. Plan. Stud. 2019, 24, 13–22. [Google Scholar] [CrossRef]
  4. Innes, J.E.; Booher, D.E. Reframing public participation: Strategies for the 21st century. Plan. Theory Pract. 2004, 5, 419–436. [Google Scholar] [CrossRef]
  5. Batty, M. Planning support systems and the new logic of computation. Reg. Dev. Dialogue 1995, 16, 1–17. [Google Scholar]
  6. Geertman, S. PSS: Beyond the implementation gap. Transp. Res. Part A Policy Pract. 2017, 104, 70–76. [Google Scholar] [CrossRef]
  7. Pelzer, P.; Geertman, S.; van der Heijden, R.; Rouwette, E. The added value of Planning Support Systems: A practitioner’s perspective. Comput. Environ. Urban Syst. 2014, 48, 16–27. [Google Scholar] [CrossRef]
  8. Flacke, J.; de Boer, C.; van den Bosch, F.; Pfeffer, K. Interactive Planning Support Systems with Citizens: Lessons Learned from Renewable Energy. In Handbook of Planning Support Science; Geertman, S., Stillwell, J., Eds.; Edward Elgar Publishing: Cheltenham, UK, 2020; pp. 482–504. [Google Scholar]
  9. Flacke, J.; de Boer, C. An Interactive Planning Support Tool for Addressing Social Acceptance of Renewable Energy Projects in The Netherlands. IJGI 2017, 6, 313. [Google Scholar] [CrossRef][Green Version]
  10. Arciniegas, G.; Janssen, R. Spatial decision support for collaborative land use planning workshops. Landsc. Urban Plan. 2012, 107, 332–342. [Google Scholar] [CrossRef]
  11. Pelzer, P.; Arciniegas, G.; Geertman, S.; de Kroes, J. Using MapTable® to learn about sustainable urban development. In Planning Support Systems for Sustainable Urban Development; Geertman, S., Toppen, F., Stillwell, J., Eds.; Springer: Berlin/Heidelberg, Germany, 2013; pp. 167–186. [Google Scholar]
  12. Shrestha, R.; Köckler, H.; Flacke, J.; Martinez, J.; van Maarseveen, M. Interactive Knowledge Co-Production and Integration for Healthy Urban Development. Sustainability 2017, 9, 1945. [Google Scholar] [CrossRef][Green Version]
  13. Russo, P.; Lanzilotti, R.; Costabile, M.F.; Pettit, C.J. Towards satisfying practitioners in using Planning Support Systems. Comput. Environ. Urban Syst. 2018, 67, 9–20. [Google Scholar] [CrossRef]
  14. te Brömmelstroet, M. PSS are more user-friendly, but are they also increasingly useful? Transp. Res. Part A Policy Pract. 2017, 91, 96–107. [Google Scholar] [CrossRef][Green Version]
  15. Pelzer, P. Usefulness of planning support systems: A conceptual framework and an empirical illustration. Transp. Res. Part A Policy Pract. 2017, 104, 84–95. [Google Scholar] [CrossRef]
  16. te Brömmelstroet, M. Performance of Planning Support Systems. Comput. Environ. Urban Syst. 2013, 41, 299–308. [Google Scholar] [CrossRef]
  17. Coors, V.; Jasnoch, U.; Jung, V. Using the virtual table as an interaction platform for collaborative urban planning. Comput. Graph. 1999, 23, 487–496. [Google Scholar] [CrossRef]
  18. Ben-Joseph, E.; Ishii, H.; Underkoffler, J.; Piper, B.; Yeung, L. Urban simulation and the luminous planning table: Briding the gap between the digital and the tangible. J. Plan. Educ. Res. 2001, 21, 195–202. [Google Scholar] [CrossRef]
  19. Hopkins, L.D.; Ramanathan, R.; Pallathucheril, V.G. Interface for a sketch-planning workbench. Comput. Environ. Urban Syst. 2004, 28, 653–666. [Google Scholar] [CrossRef]
  20. Bulens, J.; Ligtenberg, A. The MapTable, an interactive instrument for spatial planning design processes. In Proceedings of the 9th AGILE Conference on Geographic Information Science, Visegrád, Hungary, 20–22 April 2006. [Google Scholar]
  21. Voskamp, I.; van de Ven, F. Planning support system for climate adaptation: Composing effective sets of blue-green measures to reduce urban vulnerability to extreme weather events. Build. Environ. 2015, 83, 159–167. [Google Scholar] [CrossRef]
  22. Leao, S.Z.; Troy, L.; Lieske, S.N.; Randolph, B.; Pettit, C. A GIS based planning support system for assessing financial feasibility of urban redevelopment. GeoJournal 2018, 83, 1373–1392. [Google Scholar] [CrossRef]
  23. Chokshi, A.; Seyed, T.; Marinho Rodrigues, F.; Maurer, F. ePlan Multi-Surface: A Multi-Surface Environment for Emergency Response Planning Exercises. In Proceedings of the 9th ACM International Conference on Interactive Tabletops and Surfaces, Dresden, Germany, 16–19 November 2014; Dachselt, R., Graham, N., Hornbaek, K., Nacenta, M., Eds.; ACM: New York, NY, USA, 2014; pp. 219–228. [Google Scholar]
  24. Paelke, V.; Nebe, K.; Geiger, C.; Klompmaker, F.; Fischer, H. Multi-Modal, Multi-Touch Interaction with Maps in Disaster Management Applications. In Proceedings of the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Melbourne, Australia, 5 August–1 September 2012; pp. 55–60. [Google Scholar]
  25. Vonk, G.; Ligtenberg, A. Socio-technical PSS development to improve functionality and usability—Sketch planning using a Maptable. Landsc. Urban Plan. 2010, 94, 166–174. [Google Scholar] [CrossRef]
  26. Dias, E.; Linde, M.; Rafiee, A.; Koomen, E.; Scholten, H. Beauty and Brains: Integrating Easy Spatial Design and Advanced Urban Sustainability Models. In Planning Support Systems for Sustainable Urban Development; Geertman, S., Toppen, F., Stillwell, J., Eds.; Springer: Berlin/Heidelberg, Germany, 2013; pp. 469–484. [Google Scholar]
  27. Ryall, K.; Morris, M.R.; Everitt, K.; Forlines, C.; Shen, C. Experiences with and Observations of Direct-Touch Tabletops. In Proceedings of the 1st IEEE International Workshop on Horizontal Interactive Human-Computer Systems, Adelaide, Australia, 5–7 January 2006. [Google Scholar]
  28. Doeweling, S.; Tahiri, T.; Riemann, J.; Muehlhaeuser, M. Collaborative Interaction with Geospatial Data—A Comparison of Paper Maps, Desktop GIS and Interactive Tabletops. In Collaboration Meets Interactive Spaces; Anslow, C., Campos, P., Jorge, J., Eds.; Springer: Cham, Switzerland, 2016; pp. 319–348. [Google Scholar]
  29. Rogers, Y.; Lindley, S. Collaborating around vertical and horizontal large interactive displays: Which way is best? Interact. Comput. 2004, 16, 1133–1152. [Google Scholar] [CrossRef]
  30. Tong, L.; Tabard, A.; George, S.; Serna, A. Horizontal vs. Vertical: How the Orientation of a Large Interactive Surface Impacts Collaboration in Multi-surface Environments. In Proceedings of the IFIP Conference on Human-Computer Interaction, Bombay, India, 25–29 September 2017; pp. 202–222. [Google Scholar]
  31. Simon, H.A. The New Science of Management Decision; Harper: New York, NY, USA, 1960. [Google Scholar]
  32. Greiving, S.; Fleischhauer, M. Spatial planning response towards natural and technological hazards. Geol. Surv. Finl. 2006, 42, 109–123. [Google Scholar]
  33. Zucca, A.; Sharifi, A.M.; Fabbri, A.G. Application of spatial multi-criteria analysis to site selection for a local park: A case study in the Bergamo Province, Italy. J. Environ. Manag. 2008, 88, 752–769. [Google Scholar] [CrossRef] [PubMed]
  34. Goodspeed, R.; Pelzer, P. Organizing, Facilitating, and Evaluating PSS Workshops. In Handbook of Planning Support Science; Geertman, S., Stillwell, J., Eds.; Edward Elgar Publishing: Cheltenham, UK, 2020. [Google Scholar]
  35. McEvoy, S.; van de Ven, F.H.; Blind, M.W.; Slinger, J.H. Planning support tools and their effects in participatory urban adaptation workshops. J. Environ. Manag. 2018, 207, 319–333. [Google Scholar] [CrossRef] [PubMed]
  36. Eikelboom, T.; Janssen, R. Collaborative use of geodesign tools to support decision-making on adaptation to climate change. Mitig. Adapt. Strat. Glob. Chang. 2017, 22, 247–266. [Google Scholar] [CrossRef][Green Version]
  37. Pelzer, P.; Arciniegas, G.; Geertman, S.; Lenferink, S. Planning Support Systems and Task-Technology Fit: A Comparative Case Study. Appl. Spat. Anal. 2015, 8, 155–175. [Google Scholar] [CrossRef][Green Version]
  38. Janssen, R.; Eikelboom, T.; Verhoeven, J.; Brouns, K. Using Geodesign to Develop a Spatial Adaptation Strategy for Friesland: In Geodesign by Integrating Design and Geospatial Sciences; Springer: Cham, Switzerland, 2014; pp. 103–116. [Google Scholar]
  39. Geertman, S.; Stillwell, J. (Eds.) Handbook of Planning Support Science; Edward Elgar Publishing: Cheltenham, UK, 2020. [Google Scholar]
  40. Brown, G.; Chin, S.Y.W. Assessing the Effectiveness of Public Participation in Neighbourhood Planning. Plan. Pract. Res. 2013, 28, 563–588. [Google Scholar] [CrossRef]
  41. Laurian, L.; Shaw, M.M. Evaluation of Public Participation. J. Plan. Educ. Res. 2009, 28, 293–309. [Google Scholar] [CrossRef]
  42. Gunton, T.; Day, J. The theory and practice of collaborative planning in resource and environmental management. Environments 2003, 31, 5–19. [Google Scholar]
  43. Rowe, G.; Frewer, L.J. Public Participation Methods: A Framework for Evaluation. Sci. Technol. Hum. Values 2000, 25, 3–29. [Google Scholar] [CrossRef]
  44. Mannarini, T.; Talò, C. Evaluating public participation: Instruments and implications for citizen involvement. Community Dev. 2013, 44, 239–256. [Google Scholar] [CrossRef]
  45. Russo, P.; Costabile, M.; Lanzilotti, R.; Pettit, C. Usability of Planning Support Systems: An Evaluation Framework. In Planning Support Systems and Smart Cities; Geertman, S., Ferreira, J., Goodspeed, R., Stillwell, J., Eds.; Springer: Heidelberg, Germany, 2015; pp. 337–354. [Google Scholar]
  46. Pelzer, P.; Geertman, S.; van der Heijden, R. A comparison of the perceived added value of PSS applications in group settings. Comput. Environ. Urban Syst. 2016, 56, 25–35. [Google Scholar] [CrossRef]
  47. Arciniegas, G.; Janssen, R.; Rietveld, P. Effectiveness of collaborative map-based decision support tools: Results of an experiment. Environ. Model. Softw. 2013, 39, 159–175. [Google Scholar] [CrossRef]
  48. Siebenhüner, B. The role of social learning on the road to sustainability. In Governance and Sustainability; Routledge: London, UK, 2005; pp. 85–99. [Google Scholar]
  49. Shrestha, R.; Flacke, J.; Martinez, J.A.; van Maarseveen, M. Interactive Cumulative Burden Assessment: Engaging Stakeholders in an Adaptive, Participatory and Transdisciplinary Approach. IJERPH 2018, 15, 260. [Google Scholar] [CrossRef] [PubMed][Green Version]
  50. Rouwette, E.A.J.A.; Vennix, J.A.M.; van Mullekom, T. Group model building effectiveness: A review of assessment studies. Syst. Dyn. Rev. 2002, 18, 5–45. [Google Scholar] [CrossRef]
  51. Innes, J.E.; Booher, D.E. Consensus Building and Complex Adaptive Systems: A Framework for Evaluating Collaborative Planning. J. Am. Plan. Assoc. 1999, 65, 412–423. [Google Scholar] [CrossRef]
  52. Corburn, J. Healthy City Planning: From Neighbourhood to National Health Equity; Routledge: New York, NY, USA, 2013. [Google Scholar]
  53. Argyris, C.; Schön, D. Organizational Learning: A Theory of Action Perspective; Addison Wesley: New York, NY, USA, 1979. [Google Scholar]
  54. Pfeffer, K.; Baud, I.; Denis, E.; Scott, D.; Sydenstricker-Neto, J. Participatory spatial knowledge management tools. Inf. Commun. Soc. 2013, 16, 258–285. [Google Scholar] [CrossRef]
  55. Jacobs, C.; Koomen, E. A GIS-assisted method for defining an integrated vision of a climate proof Groningen. In Proceedings of the 13th AGILE International Conference on Geographic Information Science, Guimarães, Portugal, 10–14 May 2010. [Google Scholar]
  56. Bhandari, C.; Sharma, S.; Bishop, I.D.; Pettit, C. Visualizing future biolinks using a touch table—New dimensions in planning. In Proceedings of the Joint International Conference on Theory, Data Handling and Modelling in GeoSpatial Information Science, Hong Kong, China, 26–28 May 2010. [Google Scholar]
  57. Sharma, S.; Pettit, C.; Bishop, I.; Chan, P.; Sheth, F. An Online Landscape Object Library to Support Interactive Landscape Planning. Future Internet 2011, 3, 319–343. [Google Scholar] [CrossRef]
  58. Ligtenberg, A.; de Vries, B.; Vreenegoor, R.; Bulens, J. SimLandScape, a sketching tool for collaborative spatial planning. Urban Des. Int. 2011, 16, 7–18. [Google Scholar] [CrossRef]
  59. Slager, K.; Ligtenberg, A.; de Vries, B.; de Waard, R. Simlandscape: Serious gaming in participatory spatial planning. In Proceedings of the 10th AGILE International Conference on Geographic Information Science, Allborg, Denmark, 8–11 May 2007. [Google Scholar]
  60. Alexander, K.A.; Janssen, R.; Arciniegas, G.; O’Higgins, T.G.; Eikelboom, T.; Wilding, T.A.; Krkosek, M. Interactive Marine Spatial Planning: Siting Tidal Energy Arrays around the Mull of Kintyre. PLoS ONE 2012, 7, e30031. [Google Scholar] [CrossRef]
  61. Maquil, V.; Leopold, U.; de Sousa, L.M.; Schwartz, L.; Tobias, E. Towards a framework for geospatial tangible user interfaces in collaborative urban planning. J. Geogr. Syst. 2018, 20, 185–206. [Google Scholar] [CrossRef]
  62. Lenferink, S.; Arciniegas Lopez, G.; Samsura, A.; Carton, L. Integrating Geodesign and Game Experiments for Negotiating Urban Development. 2018. Available online: (accessed on 15 February 2018).
  63. Conniff, A.; Colley, K.; Irvine, K. Exploring Landscape Engagement through a Participatory Touch Table Approach. Soc. Sci. 2017, 6, 118. [Google Scholar] [CrossRef][Green Version]
  64. Boulange, C.; Pettit, C.; Giles-Corti, B. The Walkability Planning Support System: An Evidence-Based Tool to Design Healthy Communities; Geertman, S., Ed.; Springer: Cham, Switzerland, 2017; pp. 153–165. [Google Scholar]
  65. Boulange, C.; Pettit, C.; Gunn, L.D.; Giles-Corti, B.; Badland, H. Improving planning analysis and decision making: The development and application of a Walkability Planning Support System. J. Transp. Geogr. 2018, 69, 129–137. [Google Scholar] [CrossRef]
  66. Pelzer, P.; Goodspeed, R.; te Brömmelstroet, M. Facilitating PSS Workshops: A Conceptual Framework and Findings from Interviews with Facilitators. In Planning Support Systems and Smart Cities; Geertman, S., Ferreira, J., Goodspeed, R., Stillwell, J., Eds.; Springer: Heidelberg, Germany, 2015; pp. 355–369. [Google Scholar]
  67. Phoenix; Geodan. Available online: (accessed on 17 December 2019).
  68. CommunityViz Software for Planners. Available online: (accessed on 17 December 2019).
  69. Armitage, D.; Marschke, M.; Plummer, R. Adaptive co-management and the paradox of learning. Glob. Environ. Chang. 2008, 18, 86–98. [Google Scholar] [CrossRef]
  70. UN Habitat. Urbanization and Development: Emerging Futures: World Cities Report 2016; UN Habitat: Nairobi, Kenya, 2016. [Google Scholar]
  71. Gemeente Enschede. Enschede Wekt op: De Inwoners Hebben Gesproken; Nu Samen Verder: Enschede, The Netherlands, 2017. [Google Scholar]
  72. te Brömmelstroet, M.; Schrijnen, P.M. From Planning Support Systems to Mediated Planning Support: A Structured Dialogue to Overcome the Implementation Gap. Environ. Plan. B Plan. Des. 2010, 37, 3–20. [Google Scholar] [CrossRef]
  73. Steiniger, S.; Hunter, A.J. The 2012 free and open source GIS software map—A guide to facilitate research, development, and adoption. Comput. Environ. Urban Syst. 2013, 39, 136–150. [Google Scholar] [CrossRef]
  74. Onyimbi, J.; Koeva, M.; Flacke, J. Public Participation Using 3D Web-Based City Models: Opportunities for E-Participation in Kisumu, Kenya. IJGI 2018, 7, 454. [Google Scholar] [CrossRef][Green Version]
  75. Vonk, G.; Geertman, S.; Schot, P. Bottlenecks blocking widespread usage of planning support systems. Environ. Plan. 2005, 37, 909–924. [Google Scholar] [CrossRef][Green Version]
  76. Fung, A. Putting the Public Back into Governance: The Challenges of Citizen Participation and Its Future. Public Admin. Rev. 2015, 75, 513–522. [Google Scholar] [CrossRef]
  77. Bovaird, T.; Loeffler, E. From Engagement to Co-production: The Contribution of Users and Communities to Outcomes and Public Value. Voluntas 2012, 23, 1119–1138. [Google Scholar] [CrossRef]
  78. Matosin, N.; Frank, E.; Engel, M.; Lum, J.S.; Newell, K.A. Negativity towards negative results: A discussion of the disconnect between scientific worth and scientific culture. Dis. Models Mech. 2014, 7, 171–173. [Google Scholar] [CrossRef] [PubMed][Green Version]
  79. Knight, J. Null and void. Nature 2003, 422, 554–555. [Google Scholar] [CrossRef] [PubMed]
  80. Fanelli, D. Negative results are disappearing from most disciplines and countries. Scientometrics 2012, 90, 891–904. [Google Scholar] [CrossRef]
  81. Gwet, K. Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement among Raters; Advanced Analytics, LLC: Boca Raton, FL, USA, 2014. [Google Scholar]
  82. Te Brömmelstroet, M. Towards a pragmatic research agenda for the PSS domain. Transp. Res. Part A Policy Pract. 2017, 104, 77–83. [Google Scholar] [CrossRef][Green Version]
  83. Aguilar, R.; Flacke, J.; Pfeffer, K. Towards supporting collaborative spatial planning: Conceptualisation of a maptable tool through user stories. ISPRS Int. J. Geo-Inf. 2020, 9, 29. [Google Scholar] [CrossRef][Green Version]
  84. Brhel, M.; Meth, H.; Maedche, A.; Werder, K. Exploring principles of user-centered agile software development: A literature review. Inf. Softw. Technol. 2015, 61, 163–181. [Google Scholar] [CrossRef]
  85. Champlin, C.J. Contextualizing Planning Support (Systems): Co-Designing to Fit the Dynamics of Spatial Strategy Making. Ph.D. Thesis, Universiteit Twente, Enschede, The Netherlands, 2019; p. 172. [Google Scholar] [CrossRef][Green Version]
  86. Van de Ven, F.H.; Snep, R.P.; Koole, S.; Brolsma, R.; van der Brugge, R.; Spijker, J.; Vergroesen, T. Adaptation Planning Support Toolbox: Measurable performance information based tools for co-creation of resilient, ecosystem-based urban plans with urban designers, decision-makers and stakeholders. Environ. Sci. Policy 2016, 66, 427–436. [Google Scholar] [CrossRef][Green Version]
Figure 1. Stakeholders collaborating on a maptable (Source J. Flacke).
Figure 1. Stakeholders collaborating on a maptable (Source J. Flacke).
Ijgi 09 00049 g001
Figure 2. Review methodology.
Figure 2. Review methodology.
Ijgi 09 00049 g002
Table 1. Framework.
Table 1. Framework.
Dimensions, Aspects and CriteriaDefinition in a PSS Workshop ContextSources
Process: Communication and collaboration (C)
Transparent (tr)Information about issues and process is available, clarity on model assumptions is given[40,41]
Fair (fa)Dialogue is free from bias, no group or single participant is dominating the workshop, and all participants are respectful with each other[40,41]
Inclusive (in)All participants are given equal opportunities to actively participate in the discussion[40,41,44]
Efficient (ef)Same and more tasks can be conducted with lower investments and/or in less time[7,16]
Process: Social learning (L)
Awareness raising (aw)Participants are informed and increase their knowledge about an issue[9,41]
Single loop learning (sl)Participants are changing behaviour to address a challenging situation[12,49]
Double loop learning (dl)Participants are reflecting underlying assumptions and values[12,49]
Outcomes: Issue related outcomes
Knowledge created/integrated (kn)Participants tacit/experiential knowledge is added[12,49]
Consensus achieved (co)Acceptable solutions are found based on mutual understanding[7,40,41]
Understanding of the problem improved (un)Participants can understand the given information and material (un)[14,16]
Quality of decision improved (qa)Decision integrates broad knowledge base and public input, New aspects added[41]
Outcomes: Social outcomes
Ownership/commitment (ow)Participants are committed to the outcome/plan[16,50]
Mutual understanding (mu)Participants understand each other’s perspectives and issues[14,16,40,41]
Community building (cb)Development of new collaborations, improved social cohesion[51]
Satisfaction (sa)High satisfaction among participants, participants are satisfied with process and outcomes[14,16,40,41]
Table 2. Studies to be included in the review.
Table 2. Studies to be included in the review.
IDInteractive PSSYearPurposeReferences
1MapTable2006Collaborative design water management measures along a river[20]
2Touch Table2010Integration of sectoral climate change adaptation measures[55]
3Landscape Constructor System20113D design and visualization of ecological corridor for forest regeneration[56,57]
4SimLandScape2011Integration of design and planning to facilitate a collaborative process[58,59]
5Interactive decision support tool2012Involvement of stakeholders in the entire planning process of analysis, design, choice[10,60]
6MapTable PSS2013Improved communication among stakeholders during different planning stages[11,37]
7Interactive geodesign tool2014Design of three spatial adaptation strategy for water management in peat meadow area[38]
8Geospatial Tangible User Interface (GTUI)2015Support of novel collaborative planning concepts[61]
9Adaptation Support Tool (AST)2016Collaborative design of green and blue adaptation measures[21]
10geodesign simulation game2016Negotiation of the supply of development sites at a regional level[62]
11Interactive Spatial Understanding Support System (ISUSS)2017Support of interactive knowledge co-production among stakeholders in an environmental health context[12]
12GPS Tracking and Maptable tool2017Exploration of spatial behaviour in the local landscape[63]
13Collaborative Location and Allocation Gaming Environment (COLLAGE)2017Collaborative allocation of renewable energy projects in a city[9]
14Cumulative burden assessment (CuBA)2018Collaborative assessment of multiple environmental burdens affecting health[51]
15Urban Redevelopment Financial Feasibility (URFF)2018Development different scenarios of urban redevelopment[22]
16Walkability PSS2018Exploration and design built environment scenarios regarding their impacts on walkability[64,65]
Table 3. Planning Support Systems (PSS) Review criteria.
Table 3. Planning Support Systems (PSS) Review criteria.
ApplicationCase studies, locationCities/regions and countries where the tool has been applied to
Sector/field of applicationThe sector/field of application the tool is developed for and used in within the domain of spatial planning
ScaleThe planning level in which the tool has been used, from national to sub-local/neighbourhood level
Goal of the applicationThe goal of the participatory activity, what the stakeholders are asked to do using the interactive PSS
ToolHardware setup and system architectureHardware components of tool, next to the maptable, e.g., second screen, and system architecture, e.g., stand alone or server client system
SoftwareSoftware used on the maptable
Main GIS functionsMain GIS function included in the model, e.g., drawing, annotating, multi criteria analysis, scenario analysis, 3D visualization, etc.
External model(s) coupledWhether an external model (e.g., simulation model) is coupled to the tool
Stakeholders involved in tool developmentInvolvement of stakeholders in the development of the tool
Participatory ProcessPlanning phasePhases of a typical planning process that the tool is used in: problem analysis, design, choice [31]
Stakeholders/Participants(Type of) stakeholders participating in the process (e.g., researchers, practitioners, citizens)
Participatory processWhat phases or steps the participants are going through during the participatory activity
Other participatory stepsWhether other (also non-digital) steps and methods are included in the participatory activity, e.g., the drawing of a rich picture [12]
Process facilitationInvolvement of facilitator, moderator, or chauffeur in the participatory activity [66]
Process evaluationMethods applied to evaluate workshop process and/or outcomes
Table 4. Hypothesised and measured impacts on participation in the reviewed studies (N = 16).
Table 4. Hypothesised and measured impacts on participation in the reviewed studies (N = 16).
Evaluation Dimension and CriteriaHypothesized ImpactsPositive Measured ImpactsNegative Measured Impacts
Process: Communication/collaboration
Transparent (tr)2 111
Fair (fa)422
Inclusive (in)272
Efficient (ef)342
Communication/collaboration unspecific (C)12123
Process: Social learning
Awareness raising (aw)220
Single loop learning (sl)120
Double loop learning (dl)110
Social learning, unspecific (L)451
Outcome: Issue related outcomes
Knowledge created/integrated (kn)960
Consensus achieved (co)362
Problem understanding improved (un)480
Quality of decision improved (qa)243
Outcome: Social outcomes
Ownership/commitment (ow)100
Mutual understanding (mu)290
Community building (cb)000
Satisfaction (sa)230
1 numbers of studies (N = 16) that have hypothesized or measured a certain or dimension of participation.

Share and Cite

MDPI and ACS Style

Flacke, J.; Shrestha, R.; Aguilar, R. Strengthening Participation Using Interactive Planning Support Systems: A Systematic Review. ISPRS Int. J. Geo-Inf. 2020, 9, 49.

AMA Style

Flacke J, Shrestha R, Aguilar R. Strengthening Participation Using Interactive Planning Support Systems: A Systematic Review. ISPRS International Journal of Geo-Information. 2020; 9(1):49.

Chicago/Turabian Style

Flacke, Johannes, Rehana Shrestha, and Rosa Aguilar. 2020. "Strengthening Participation Using Interactive Planning Support Systems: A Systematic Review" ISPRS International Journal of Geo-Information 9, no. 1: 49.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop