Strengthening Participation Using Interactive Planning Support Systems: A Systematic Review

Abstract

Interactive Planning Support Systems (PSS) implemented on a maptable are deemed suitable to support participatory planning processes. They are supposed to facilitate exchange of knowledge between stakeholders, consensus building among them, and group-learning processes. In this systematic review, based on 16 case studies using interactive PSS, we analyze how these have contributed to the goal of strengthening stakeholder participation. To this end, we first elicit details of the interactive PSS and the related participatory processes. In the second step, we analyze how and what the studies report, as the impacts on participation. Results show that tools and applications have become more sophisticated over time and goals of the studies changed from collaboratively designing interventions to observing and understanding how the application of such tools contributes to improved plan outcomes and group-based learning. All interactive PSS succeeded to facilitate intensive stakeholder collaboration. However, many studies lack a proper framework for investigating its impacts on participation and therefore assess these rather incidentally based on implicit assumptions. Thus, a significant outcome of this review is an evaluation framework, which allows the structural assessment of the impacts of interactive PSS on stakeholder participation.

1. Introduction

With the communicative turn [1], planning theory moved its focus from “a preoccupation with the distribution of material resources” to a “process of working out how to coexist in shared space” [2] (p. 219). Since then, collaborative planning, understood as an interactive and interpretative process involving a dialogue between stakeholders [2] (p. 221), has become the major paradigm guiding the practice of spatial planning and strategy making. However, the growing complexity of multidimensional planning tasks [3] calls for other forms of public participation, i.e., forms that go beyond what is legally required. These should enable “a multiple-way set of interactions among citizens and other players who together produce outcomes” [4] (p. 419). Closely related, Planning Support Systems (PSS), defined as “geo-information-technology-based instruments that are dedicated to supporting those involved in planning in the performance of their specific tasks” [5], are being implemented as instruments attuned to support specific planning tasks [6].

A particular type of PSS aiming to enable the collaboration of stakeholders are PSS developed for interactive surfaces, so-called interactive PSS. These interactive PSS are hardware systems in the form of a touch-sensitive table, a maptable, combined with geospatial mapping and analytical tools to support group processes [7]. Such interactive PSS provide a platform for discussing spatial problems or evaluating planned alternatives using a shared map interface to facilitate interaction among stakeholders [8]. Through their interactive nature and user-friendly interface a maptable might facilitate exchange of knowledge and preferences between stakeholders [9], consensus building among them [10], and group learning about complex problems [11,12], among others.

Since almost 15 years, various applications of such interactive PSS have been developed and implemented in case studies, often related to urban planning. These case studies, their specific contexts, tools implemented and participatory processes supported are often well reported. However, many studies focus rather on the evaluation of the usability [13] or user friendliness [14] of the PSS, i.e., the functioning of the tool [15], then on its usefulness, i.e., the impact of the use of such tools on the planning quality, though the usability of a PSS is rather a means than a goal in itself [16]. What is missing is a systematic review of case studies applying interactive PSS regarding their impacts on stakeholder participation.

We analyze in this review, based on case studies reported in interactive PSS publications, how such PSS have contributed to the goal of strengthening stakeholder participation. To this end, we first review existing interactive PSS and elicit details of the applications and cases studies, the PSS tools developed and the participatory processes they were used in. In the second step we then systematically analyze how and what the various studies report as the impacts of the use of the interactive PSS on participation, based on a framework for evaluating participation that we develop beforehand.

Accordingly, the paper is structured as follows: In Section 2 we provide an account of interactive PSS applications, tools and definitions and their use in stakeholder workshops. In Section 3 we develop a framework for evaluating impacts of interactive PSS on stakeholder participation. Section 4 elaborates the methodology applied in the review. Section 5 provides the results of the review analysis, i.e., the review of the interactive PSS applications, tools, and participatory processes (Section 5.1), and the reported impacts of interactive PSS on the various aspects and criteria of participation (Section 5.2). Section 6 discusses the results gained in the review before Section 7 concludes with key findings and further research needs.

2. Interactive PSS Enhancing Stakeholder Participation in Planning

Conceptual and technological research on maptables dates back to the turn of the century. Coors et al. [17] presented a first prototype of a virtual table composed of an interaction interface and 3D-GIS linked to it. In 2001, the MIT presented its Luminous Planning Table (LPT), a prototype developed as a “new form of information delivery that is direct and easily understood by professionals and laypersons” [18] (p. 195). Hopkins et al. [19] presented a so-called sketch-planning workbench implemented on a large-scale touch-sensitive screen that used gestures for drawing, annotating, and manipulating ideas during a collaborative sketch planning session.

The first case study of interactive PSS implemented in stakeholders workshops dates back to 2006 [20]. Since then various applications of interactive PSS have been developed and tested in various sectors of urban and regional planning, including sustainable urban development [11], climate change adaptation [21], energy transition [9], environmental health [12], and urban redevelopment [22], among others. Next to that, a couple of studies applying maptables in the field of disaster risk management can be found [23,24].

2.1. Interactive PSS Tools

An interactive PSS tool typically consists of a large-scale horizontal, touch-sensitive screen, that stakeholders can stand around and interact with, and a geospatial tool that enables the interaction (Figure 1). Sometimes a second, smaller screen is added to the setup to visualize the results of stakeholder’s interaction or other information, e.g., a 3D model. While over the last ten years, technological innovations have immensely improved the availability and affordability of large screen touchscreen as the key hardware for interactive PSS, suitable geospatial tools particularly made for interactive PSS applications are still hardly available. The development of interactive PSS tools is typically done by researchers for specific case studies and contexts, in some cases following a socio-technical approach [25] in order to increase their usefulness [15].

Figure 1. Stakeholders collaborating on a maptable (Source J. Flacke).

Proven benefit of an interactive PSS compared to a desktop PSS is that the maptable has a positive influence on working styles and group dynamics and thereby allows an enhanced interaction between stakeholders. Therefore, such interactive PSS are particularly suited to support participatory activities within a planning process. Dias et al. [26] describe the use of such interactive PSS in participatory planning processes as a promising way to bridge the gap between the creative design process of architects and urban designers and the more analytical process of planners. Ryall et al. [27] report the use of such applications “less intimidating, and often more playful than a traditional computer desktop environment”. Doeweling et al. [28] compared the use of classical paper maps, desktop GIS and interactive maptables in the context of stakeholder collaboration during crisis management. The authors found that a maptable-based solution can provide significant advantages when multiple users with different roles collaborate in sense making and planning tasks. Finally, the horizontal setup of a maptable revealed significant benefits for group collaboration compared to a vertical setup of the same device [29,30].

2.2. Planning Processes and Interactive PSS Workshops

A typical planning process can be structured into the phases of intelligence, design and choice [31]. The intelligence phase often comprises a baseline and problem analysis, in the design phase typically alternative plans or solutions are designed, while in the choice phase a decision is made on which plan alternative to implement [32]. Theoretically, stakeholders might be involved in any or even multiple of these phases of a planning process, e.g., by contributing the problem analysis, designing alternative plans, or evaluating the different alternatives [33].

The application of interactive PSS tools is typically done in PSS workshops [34] that often relate to a specific step or task within such a planning process. Two types of workshops can be distinguished here, controlled experiments and close to real-world stakeholder workshops. In controlled experiments, tasks often involve students playing a role of stakeholders [34]. Such experiments allow to test specific questions with respect to the usability of the tool, a comparative analysis of different tools [28,35], or tools being tested under changing conditions or with different user groups [36]. Moreover, regularly larger samples for statistical analysis are achieved.

On the other hand, in close to real world stakeholder workshops, tests provide richer information on their added value for the planning process and outcomes. However, those workshops are more complex to set up and implement, and include a certain level of uncertainty. Ideally, controlled experiments with a PSS are conducted prior to using it in stakeholder workshops. Finally, both types of workshops benefit from a high degree of facilitation, including a moderator steering the discussion and a chauffeur supporting the use of the PSS [37].

Methods for the evaluation of the use of interactive PSS in stakeholder workshops range from observations, interviews with participants, surveys based on questionnaire, to audio or video recording of stakeholder sessions. Often various methods are combined in order to verify the consistency of findings based on multiple sources via triangulation. Janssen et al. [38] conducted participant’s surveys before and after the workshops in order to explore how perceptions and preferences changed throughout a planning session.

3. Impacts of Interactive PSS on Participation

In order to analyze to what extent interactive PSS tools and applications have contributed to the goal of strengthening participation during PSS workshops, a framework for evaluating participation is needed. Such framework can be built from two partly separated scientific debates: First, the evaluation of public participation, which is an essential element of planning practice; second, the evaluation of PSS in the context of planning support science [39].

The evaluation of public participation within a planning context essentially looks at the effectiveness [40] or success or failure [41] of the participatory activities and processes. Gunton and Day [42] (p. 9) identified four key evaluative criteria for assessing participatory practices, which are: “(1) Success in reaching an agreement, (2) efficiency of the collaborative process relative to alternative processes, (3) satisfaction of stakeholders with the process and outcome, and (4) achievement of other social capital benefits such as improved relationships among stakeholders and enhanced stakeholder skills and knowledge”. More elaborated evaluation frameworks were developed by Rowe and Frewer [43], Laurian and Shaw [41], Brown and Wei Chin [40], and Mannieri and Talo [44]. Commonly, these frameworks distinguish between process criteria, that refer to the effective construction and implementation of a participatory activity, and acceptance or outcome criteria that refer to the broad range of results obtained from a participatory activity [40,43]. The frameworks include a number of—partly similar—process and outcome criteria. Depending on the goals of an individual participatory activity different sets of criteria might be taken into account, because “each of these criteria suggests a different perspective on what ‘successful’ participation means” [41] (p. 296). Moreover, these criteria are not mutually exclusive, because a participatory process might “seek to promote different goals simultaneously” [41]. Finally, these criteria are developed and used independent of the individual participatory methods, e.g., public hearing, consensus conference, focus group discussion.

The evaluation of the use of PSS in participatory workshops has focused for long time rather on the usability of the PSS tools [13,45] than on its impacts on the participatory process. Only since 2010 the latter has gained attention in the scientific debate on PSS (e.g., [7,8,14,15,16,46]. However, studies that evaluate the impact of the tools on the quality of the planning process and outcomes adopt different assessment frameworks, such as performance [16], added value [7,46], or effectiveness [47], all of them being based rather on implicit assumptions than on empirically grounded outcomes [15]. These frameworks likewise distinguish process and outcome criteria, but a thorough operationalization of the criteria is often lacking. Instead, scholars investigate potential correlations between the usefulness of a PSS and certain explanatory variables manifested in the usability of the particular PSS tool in order to determine options for improvement of the PSS. For example, te Brömmelstroet [14] has investigated the correlation between the usefulness of a PSS and its user-friendliness.

The evaluation framework developed for this study (Table 1) integrates dimensions and criteria from both scientific debates. It distinguishes criteria to evaluate the participatory process as well as those to evaluate the outcomes achieved through the participatory process. Both types of evaluation criteria are meant to evaluate participatory activities conducted in a PSS workshop context.

Table 1. Framework.

Dimensions, Aspects and Criteria	Definition in a PSS Workshop Context	Sources
Process: Communication and collaboration (C)
Transparent (tr)	Information about issues and process is available, clarity on model assumptions is given	[40,41]
Fair (fa)	Dialogue is free from bias, no group or single participant is dominating the workshop, and all participants are respectful with each other	[40,41]
Inclusive (in)	All participants are given equal opportunities to actively participate in the discussion	[40,41,44]
Efficient (ef)	Same and more tasks can be conducted with lower investments and/or in less time	[7,16]
Process: Social learning (L)
Awareness raising (aw)	Participants are informed and increase their knowledge about an issue	[9,41]
Single loop learning (sl)	Participants are changing behaviour to address a challenging situation	[12,49]
Double loop learning (dl)	Participants are reflecting underlying assumptions and values	[12,49]
Outcomes: Issue related outcomes
Knowledge created/integrated (kn)	Participants tacit/experiential knowledge is added	[12,49]
Consensus achieved (co)	Acceptable solutions are found based on mutual understanding	[7,40,41]
Understanding of the problem improved (un)	Participants can understand the given information and material (un)	[14,16]
Quality of decision improved (qa)	Decision integrates broad knowledge base and public input, New aspects added	[41]
Outcomes: Social outcomes
Ownership/commitment (ow)	Participants are committed to the outcome/plan	[16,50]
Mutual understanding (mu)	Participants understand each other’s perspectives and issues	[14,16,40,41]
Community building (cb)	Development of new collaborations, improved social cohesion	[51]
Satisfaction (sa)	High satisfaction among participants, participants are satisfied with process and outcomes	[14,16,40,41]

Process criteria include on the one hand criteria assessing the quality of the communication and collaboration between stakeholders during the PSS workshop and on the other hand whether and how social learning, understood as a learning through group activities [48], happens during the process. A participatory activity enabling a good communication and collaboration between stakeholders supports what Laurian and Shaw [41] call a democratic process. Such a process should ideally be transparent ([40,41], fair [40,41], inclusive [40,41,44], and efficient ([7,16]. The two criteria of a process being inclusive and fair are closely related to each other, though the latter is sometimes overlooked in participatory practice [41]. Inclusive means that all participants are given equal opportunities to attend a workshop [44]. Fair means that all participants are given equal voice or standing [41], the dialogue is free from bias, no group or single participant is dominating the discussions, and all participants are respectful with each other [44].

Criteria for social learning during a PSS workshop evaluate the development and increase of knowledge and understanding [44] of the participants during the participatory PSS process. Different levels of social learning are to be distinguished here. Awareness raising occurs when participants are informed about issues and stakes at hand [41] and they increase their knowledge about these [9]. Single and double loop learning occurs when repeated experiences shared in group dialogues help participants to detect a mismatch between intentions and consequences [52]. Single loop learning is defined when participants are challenging their behavior to address a challenging situation, while double loop learning refers to participants reflecting underlying assumptions and values [53].

Outcome criteria are structured into issue related outcomes and social outcomes [41]. The latter is frequently neglected in PSS workshops, probably due to their singular, often project-related character. Issue related outcome criteria evaluate the broad range of results obtained from a PSS workshop. They comprise the creation and/or integration of knowledge, such as tacit, contextual-embedded, and codified knowledge [54], whether there is an enhanced understanding of the problem [12], whether a consensus is achieved, and whether the quality of the decision is improved [41]. The last one also includes the aspect of novel ideas or issues being developed (e.g., new alternative plans or interventions), that te Brömmelstroet [14] mentions. Social outcome criteria measure whether PSS workshops result in capacity building, and increased trust and social networks [41]. They include creation of commitment and ownership among the participants [16,50], the development of a mutual understanding of each other’s perspectives and issues [14,16,40,41], the initiation of community building, e.g., new collaborations or improved social cohesion [51], and whether participants are satisfied with the outcomes of the workshop [14,16,40,41].

Though separated in the framework, process and outcome criteria are closely related, because they likely affect each other [51]. For instance, achieving a common and good understanding of the problem is a prerequisite of achieving a consensus. Likewise, a fair and inclusive process is not only a prerequisite for consensus building but to a certain degree also for achieving a common and mutual understanding.

4. Review Methodology

The review is based on the analysis of studies describing interactive PSS that are used for facilitating stakeholder collaboration in urban planning related participatory activities. Figure 2 depicts the entire review methodology.

Figure 2. Review methodology.

The review includes tools and applications published in journal articles and conference proceedings. If various articles refer to the same interactive PSS tool, they are considered as one study. The search strategy for finding publications included two steps. First, we searched for publications from the year 2000 to 2018 in databases using relevant combinations of keywords plus commonly used synonyms. We then screened the obtained long list of publications and selected papers based on predefined selection criteria. Scopus was used as the search database, because it also includes publications in conference proceedings, with the following search terms:

• Keyword = maptable or one of the following synonyms: touchtable, tabletop, touch user interface, tangible user interface, tui, plus different spellings of these terms such as map table;
• AND
• Keyword = urban planning or one of the following related terms: spatial planning, strategic planning, urban design.

In total 58 papers were found in Scopus. After reading the abstract of these papers, a short list of papers to be included in the review was compiled based on the following selection criteria:

• Focus on stakeholder participation in spatial planning in general, i.e., planning related activities at any scale (national to sub-local),
• Use of the maptable to facilitate the participatory process,
• Real-world case study, i.e., applied in a real context, with real stakeholders, and addressing a real or at least realistic problem setting, i.e., a typical contemporary planning issue,
• Peer reviewed and published articles, book chapters, or conference papers,
• Interactive PSS applications, tool and workshop described in sufficient detail, so that the necessarily details for all criteria can be elicited.

The following table presents the 16 tools and papers included in the study (Table 2). All studies describe the applications of interactive PSS in stakeholder workshops to support steps in a planning process. Some studies made use of multiple case studies. Study ID 6 included one case that was not truly supporting a group process, because stakeholders had to use the table one by one.

Table 2. Studies to be included in the review.

ID	Interactive PSS	Year	Purpose	References
1	MapTable	2006	Collaborative design water management measures along a river	[20]
2	Touch Table	2010	Integration of sectoral climate change adaptation measures	[55]
3	Landscape Constructor System	2011	3D design and visualization of ecological corridor for forest regeneration	[56,57]
4	SimLandScape	2011	Integration of design and planning to facilitate a collaborative process	[58,59]
5	Interactive decision support tool	2012	Involvement of stakeholders in the entire planning process of analysis, design, choice	[10,60]
6	MapTable PSS	2013	Improved communication among stakeholders during different planning stages	[11,37]
7	Interactive geodesign tool	2014	Design of three spatial adaptation strategy for water management in peat meadow area	[38]
8	Geospatial Tangible User Interface (GTUI)	2015	Support of novel collaborative planning concepts	[61]
9	Adaptation Support Tool (AST)	2016	Collaborative design of green and blue adaptation measures	[21]
10	geodesign simulation game	2016	Negotiation of the supply of development sites at a regional level	[62]
11	Interactive Spatial Understanding Support System (ISUSS)	2017	Support of interactive knowledge co-production among stakeholders in an environmental health context	[12]
12	GPS Tracking and Maptable tool	2017	Exploration of spatial behaviour in the local landscape	[63]
13	Collaborative Location and Allocation Gaming Environment (COLLAGE)	2017	Collaborative allocation of renewable energy projects in a city	[9]
14	Cumulative burden assessment (CuBA)	2018	Collaborative assessment of multiple environmental burdens affecting health	[51]
15	Urban Redevelopment Financial Feasibility (URFF)	2018	Development different scenarios of urban redevelopment	[22]
16	Walkability PSS	2018	Exploration and design built environment scenarios regarding their impacts on walkability	[64,65]

The analysis of the studies is structured into two parts (see Figure 2). In the first part the interactive PSS are reviewed in detail, structured into three categories that are defined above: (a) interactive PSS applications, (b) interactive PSS tools, (c) participatory process. Category (a) looks at the context of the applications, i.e., the case studies, the sector, scale and the goals of the applications, category, category (b) studies the details of the hard- and software details of the interactive PSS tools and how they were developed, category (c) finally studies the characteristics of the participatory processes the interactive PSS were applied to including the methods of evaluation. The full list of review criteria, that have been defined based on a screening of all selected papers, is given below (Table 3).

Table 3. Planning Support Systems (PSS) Review criteria.

Category	Criteria	Description
Application	Case studies, location	Cities/regions and countries where the tool has been applied to
	Sector/field of application	The sector/field of application the tool is developed for and used in within the domain of spatial planning
	Scale	The planning level in which the tool has been used, from national to sub-local/neighbourhood level
	Goal of the application	The goal of the participatory activity, what the stakeholders are asked to do using the interactive PSS
Tool	Hardware setup and system architecture	Hardware components of tool, next to the maptable, e.g., second screen, and system architecture, e.g., stand alone or server client system
	Software	Software used on the maptable
	Main GIS functions	Main GIS function included in the model, e.g., drawing, annotating, multi criteria analysis, scenario analysis, 3D visualization, etc.
	External model(s) coupled	Whether an external model (e.g., simulation model) is coupled to the tool
	Stakeholders involved in tool development	Involvement of stakeholders in the development of the tool
Participatory Process	Planning phase	Phases of a typical planning process that the tool is used in: problem analysis, design, choice [31]
	Stakeholders/Participants	(Type of) stakeholders participating in the process (e.g., researchers, practitioners, citizens)
	Participatory process	What phases or steps the participants are going through during the participatory activity
	Other participatory steps	Whether other (also non-digital) steps and methods are included in the participatory activity, e.g., the drawing of a rich picture [12]
	Process facilitation	Involvement of facilitator, moderator, or chauffeur in the participatory activity [66]
	Process evaluation	Methods applied to evaluate workshop process and/or outcomes

The second part of the analysis aims at understanding how the studies covered in the review have addressed and measured impacts on stakeholder participation during the PSS workshops. In order to do so we analysed how the studies report on hypothesized or measured impacts on participation using the framework developed above (Section 3, Table 1). Statements of the papers regarding the impacts of the interactive PSS use on various aspects of participation were matched to the different evaluation criteria and classified into hypothesized or measured (see Appendix B, Table A4, Table A5, Table A6 and Table A7), depending on whether the authors aim to achieve the effects (i.e., hypothesized) or provide empirical evidence of having achieved them (i.e., measured). Where multiple statements in a study regarding the same dimension or criteria were found, these were counted as one in the final table (Table 4), because it is not relevant for the analysis how often a certain aspect of participation was mentioned. Negative statements indicating that certain dimensions of participation were not achieved were particular highlighted, because also these provide insights into the value of interactive PSS for facilitating participation. In cases where statements in the studies were not explicit but rather claimed general achieved impacts like social learning or better communication, these statements were classified in the appropriate dimension of participation.

Table 4. Hypothesised and measured impacts on participation in the reviewed studies (N = 16).

Evaluation Dimension and Criteria	Hypothesized Impacts	Positive Measured Impacts	Negative Measured Impacts
Process: Communication/collaboration
Transparent (tr)	2 1	1	1
Fair (fa)	4	2	2
Inclusive (in)	2	7	2
Efficient (ef)	3	4	2
Communication/collaboration unspecific (C)	12	12	3
Process: Social learning
Awareness raising (aw)	2	2	0
Single loop learning (sl)	1	2	0
Double loop learning (dl)	1	1	0
Social learning, unspecific (L)	4	5	1
Outcome: Issue related outcomes
Knowledge created/integrated (kn)	9	6	0
Consensus achieved (co)	3	6	2
Problem understanding improved (un)	4	8	0
Quality of decision improved (qa)	2	4	3
Outcome: Social outcomes
Ownership/commitment (ow)	1	0	0
Mutual understanding (mu)	2	9	0
Community building (cb)	0	0	0
Satisfaction (sa)	2	3	0

¹ numbers of studies (N = 16) that have hypothesized or measured a certain or dimension of participation.

The extraction of statements from the papers and their categorization into the different aspects and criteria of the framework was conducted by the first author. In order to minimize potential misinterpretation or omissions and to increase the reliability of such an intersubjective endeavour [14] the second author reviewed the results independently and added eventual missing statements. Disagreement between the two authors regarding the categorization of certain statements into the framework were resolved in a discussion of the entire team of authors.

5. Review Results

The presentation of the results is divided into two parts. First, the details of the applications and cases studies, tools developed and participatory processes they were used in are presented. Second, we analyse how and what the studies reported on the impacts on participation, based on the framework developed in Section 3.

5.1. Interactive PSS Applications, Tools, and Participatory Processes

The development of interactive PSS included in this review ranges from 2006 to 2018. Nonetheless, half of the applications have been developed and applied in PSS workshops between 2016 and 2018 (see Table 2). This shows that the use of such application has become more widespread and common in recent years. Detailed results of the review of the various interactive PSS are given in Appendix A, Table A1, Table A2 and Table A3.

The applications taken into account in this review cover a wide range of topics within the domain of spatial planning, from rural land use planning to urban redevelopment at neighbourhood level (Appendix A, Table A1). Next to common topics such as sustainable urban development, the sectors of urban health (3 applications), energy transition (4 applications) and climate change (3 applications) are predominantly addressed, which reflects typical foci of contemporary planning debates of the last 10 years. Half of the applications include at least one Dutch case study, which has to do with the fact that various research groups in the Netherlands work on the topic of interactive PSS. The goals of the applications are either related to designing spatial interventions such as adaptation measures, or to analysing the current situation and developing e.g., a good understanding of the problem and exchanging knowledge about it. Hardly any study covers the final phase of a decision-making process in which typically one option among a set of alternative designs is chosen, based on a proper evaluation.

All interactive PSS tools make use of a similar hardware setup of a maptable most often as a stand-alone system, plus in some cases an additional second screen e.g., for showing results of spatial interventions on the table in the forms of charts (Appendix A, Table A2). Maptables are operated either with special stylus pens or with fingers, only one study (ID 8) explores the use of physical objects (tags) to interact with the maptable. One PSS tool (ID 5) used a maptable that allowed distinction of inputs from different users. However, this was obviously not relevant for the results, because it was not made use of in the analysis of the workshop, nor has it prevailed in later studies.

The use of software for the tools reveals the fact that so far, hardly any software particularly made for maptables is available, with the exception of Phoenix, a participatory mapping tool [67], used in study ID 11. Most of the tools, therefore, use various ESRI ArcGIS components or an extension for the same, often CommunityViz Scenario 360 [68], which increases possibilities of dynamic user interaction with the geospatial content. Only one of the 16 tools considered here is developed based on an open source platform.

The GIS functionality mainly used by stakeholders during the interactive sessions comprises both functions for sketching and drawing features, as well as functions for calculating outcomes of certain interventions in the form of indicators, often visualized in charts. Since the limited capacity of the used GIS platforms for advanced modelling, some tools couple the GIS with a dynamic simulation model, such as a hydrologic model for calculating outcomes of interventions (ID1, ID 4b). Only three tools make use of 3D visualization for presenting spatial data. Also the inclusion of stakeholders in the development of the tools, as recommended by several scholars [13,15] in order to increase the usefulness of the tools, is hardly practised up to now, with the exception of studies IDs 4, 9, 13 (Appendix A, Table A2).

The planning phase most often addressed in the participatory processes supported by the tools is the design phase (Appendix A, Table A3). In the design phase, different stakeholders develop together measures or interventions in order to address given problems or planning challenges. In more recent applications, the problem analysis phase is also aimed at, in which the interactive PSS typically supports in gaining a thorough understanding of the given problem or an integration of knowledge prior to the design of interventions. Almost all processes involved expert stakeholders, either academics or practitioners from different domains. Only four processes (IDs 5, 7, 9, 13) aim at including also layperson, i.e., ordinary citizens, in the participatory processes.

In all processes, the participants are guided through more or less clearly defined steps or assignments that build up upon each other. Six studies combine the use of the maptable with other participatory activities such as rich picture drawing (ID 11) or GPS tracking (ID 12). The more concrete the steps are that the stakeholders have to go through, the less seems to be a need for extensive moderation. In tool ID 5 e.g., the participants first had to design suitable bicycle paths and then to evaluate land use scenarios [10]. On the other hand, for rather open processes a moderator has a relevant function to guide the process and engage the stakeholders. The use of a technical facilitator or chauffeur, who helps in handling the PSS, is common in recent participatory processes.

The early studies are not very explicit about the methods applied for evaluating the results and outcomes of the workshops or do not discuss these at all. Later studies applied a variety of methods for evaluating workshop processes and outcomes. Most common are post workshop questionnaires in which participants assessed various factors of usability of the tool as well as usefulness of the workshop process and outcomes. Seven studies combined various methods for evaluation in a sort of triangulation in order to answer various research questions or to crosscheck plausibility of findings. Only two studies applied questionnaires prior to and after the workshop in order to determine behavioural changes or learning outcomes (IDs 7 and 15).

5.2. Dimensions and Aspects of Participation Reported in the Studies

In this section, we analyse how and what the studies reported as impacts of the interactive PSS implementation on participation. The overall results are presented in Table 4 (for details see Appendix B, Table A4, Table A5, Table A6 and Table A7). The table shows how many of the in total 16 studies have explicitly reported statements related to the dimensions and criteria of participation as elaborated in the framework (Section 3, Table 1). Hypothesized and measured impacts are separated. Measured impacts are distinguished in positive and negative claims, the latter indicating aspects of participation that have not been achieved. Multiple claims that are collected for the same criteria or dimension are counted as one impact, as explained above (Section 4). Claims that could not be related to specific criteria of participation were associated with the relevant dimension of participation as an unspecific impact.

Table 4 shows that for some dimensions a huge difference between hypothesized and measured criteria of participation exists. In many studies, authors measured more detailed impacts on participation than they hypothesized upfront. In fact, several studies lack a proper framework for evaluating impacts on participation. While in some studies this was according to the authors also not the aim of the study (IDs 1, 2, 3), other studies failed to develop upfront a clear framework. Leao et al. [22] e.g., claim to test the performance and usefulness of the tool, but their study lacks an operationalization towards this goal. In other studies, the applied framework for evaluation towards participation impacts is introduced rather implicitly while explaining the questionnaire for the evaluation [61]. While the study ID 16 [65] has not stated an explicit focus on strengthening participation by using the PSS, some implicit references towards supporting participation are given, e.g., strengthening communication. On the other hand, another group of studies (IDs 6, 11, 14) developed upfront a dedicated framework for evaluating impacts on participation, which is often focusing on a specific dimension such as social learning.

Particularly in the process dimension of participation, a number of studies is making no distinction between the different criteria of participation. Both hypothesized and measured impacts are reported to support communication and collaboration (e.g., [9,64]) or trigger learning (e.g., [11,62]), without distinguishing what aspects of communication and collaboration or learning are addressed. In contrast to that, in the outcome dimension of participation impacts are more clearly distinguished both, while being hypothesized or measured. Different outcomes of a PSS workshops such as achieving a consensus is obviously much more concrete to grasp the rather vague notions of learning [69].

In total, the reviewed studies have mainly aimed at communication and collaboration as the process dimension and issue related outcomes as the outcome related dimension (Table 4). Those studies that have investigated communication and collaboration in more detail hypothesize mainly fairness (4 studies) and efficiency (3 studies) being the main goals of the interactive PSS workshops and applications. Interestingly, the criteria of inclusiveness and efficiency of the process are more often measured than hypothesized upfront. This supports the findings of the limited structured evaluation frameworks developed prior to the application. Learning during PSS workshops as an impact of participation is taken into account explicitly only in three newer studies (IDs 6, 11 14). These studies have hypothesised to investigate impacts of interactive PSS workshops on social learning and have also measured these.

Issue related outcomes are the by far most often hypothesized as well as measured dimension of participation. Nine studies aim at creating or integrating knowledge by means of interactive PSS workshop, while only six studies measure these. Three respectively four studies aim at achieving a consensus and/or increasing the problem understanding, but in six resp. 8 studies authors have measured these impacts. Interestingly, the criteria of improving the quality of the decision is only hypothesized in two studies, but measured as an issue related outcome in seven studies, four of them finding mainly positive indications, and three studies also reporting negative statements.

Finally, social outcomes are hardly addressed as hypothesized criteria in the studies. Increased ownership and commitment are hypothesized only in one study and both mutual understanding and satisfaction are only hypothesized in two studies. This shows that authors do not expect much on that dimension or do not consider that to be relevant in a PSS workshop, probably because it is often a one-time activity. On the other hand, in particular an increased mutual understanding among participants of a workshop is measured as a social outcome in nine of the studies, showing that such workshops in fact do have an effect on social aspects, at least at short term, i.e., at the time when the evaluation was done. With the same argument the creation of ownership or community building might not be considered, because these kind of impacts in fact require much more frequent and an intense working on an issue over a longer duration.

6. Discussion

6.1. Interactive PSS Applications, Tools and Processes

The interactive PSS analysed in this review reveal a large variety of applications, goals, contexts and case studies for different planning topics, illustrating the broad applicability of such applications. The more recent applications have a clear focus on issues at an urban scale [9,12,21,51,64], which on the one hand might reflect the increasing importance of urban topics in international planning debates [70]. On the other hand, this might also be due to the fact, that more small-scaled data is available nowadays enriching such urban studies and applications. Different case studies have been applied in different stages of a planning process, illustrating the potential value of these in various phases of planning. Some studies even tested the application of interactive PSS throughout different phases of a planning process [10,60], thereby demonstrating the value of such tools for supporting stakeholder participation in planning.

Nevertheless, most of the interactive PSS studies are developed and implemented in a research context, while only a minority of studies (IDs 6, 13) have been initiated through a municipality or planning agency. In the case of Pelzer et al. [11], the municipality “brought together an urban design firm, several environmental research agencies and a geo-communication firm to concretize its vision of the new neighbourhood” [11] (p. 177). The stakeholder workshops using the interactive PSS were perceived useful by all participants, but “the commercial developers (who owned most of the land) […] had not been involved in the MapTable sessions […] which led to a pause in the project execution.” [11] (p. 178). Moreover, no study discusses how the results from the PSS workshops are further used in the planning process. In the study of Flacke and de Boer [9] “the workshop results […] were summarized in a policy “document [71]. However, no follow up steps have been taken so far” [9] (p. 14). One can conclude that while the tools and applications presented in the studies might have the potential to overcome the widely discussed PSS implementation gap [6,72] any proof for that is still lacking.

The analysis of PSS tools shows that different types of hardware setups and infrastructure systems are being implemented successfully. In contrast, there is an obvious demand for the development of dedicated software tools supporting a group process on maptables. In most cases, generic and off-the-shelf GIS software augmented with certain extensions or extra functions is being used, which is not designed for tangible user interfaces, resulting in limited user friendliness of the tools. No open source tools such as QGIS, holding the promise of permanent refinement and tailor-made modification through a large user community [73], are being used. Moreover, only one study (ID 11, [12]) uses a tool that is explicitly developed for tangible user interfaces, but that tool supports basically participatory mapping studies lacking analytical functions of a PSS. In conjunction with the findings that the tool development processes hardly involve any stakeholders [14,15], we speculate that the development and implementation of such maptable based PSSs are still the job of experts and consequently the tools are far from being used in every day planning practice.

Finally, only very few tools reviewed make use of 3D visualization techniques, though this is reported to enhance participation, particularly of layperson, in planning significantly [74]. In line with that, PSS workshops reviewed here are mainly conducted involving planning experts as stakeholders, either researchers or planning practitioners, who are likely to be experienced in working with maps and 2D visualizations of spatial data [8]. Laypersons, such as ordinary citizens, are involved only in 4 of the 16 studies. From our own experience [9] this might be explained by the non-availability of user-friendly PSS tools and interfaces, e.g., tools made explicitly for tangible user interfaces, which would be easier to use also for non-experts. Vonk et al. [75] found in a study on the bottlenecks in the uptake of PSS that using such tools […] depends, among other things, upon previous experiences in planning practice. The vast potential offered by user friendly maptable applications and attractive (3D) visualization for involving particular non experts is hardly explored yet [8].

6.2. Interactive PSS Contribution to Participation

This review shows that all interactive PSS applications and tools succeeded to facilitate intensive, in some cases, enthusiastic [37] stakeholder collaboration and workshops participants reacted predominantly positive to the possibilities provided by the PSS tools. Thus, we can conclude that in all cases studies the interactive PSS has contributed to strengthening participation, as it can be seen from the results in Table 4. However, the most apparent finding is that many interactive PSS case studies do not make use of a very distinct and explicit framework in order to evaluate impacts of the interactive PSS tool on participation. Thus, impacts on participation are often being assessed rather incidentally based on implicit assumptions [15], without being underpinned by a proper conceptual or theoretical discussion. Having said this, we can conclude that many of the studies reviewed here can be characterized as studies testing the usability of the tools rather than their usefulness for the planning practice [14]. However, given the obvious strength of many of the interactive PSS for improving aspects of participation, detailed frameworks of usefulness or performance of PSS as initiated in contemporary scholarly debates [8,14,15], are needed in order to equip practical and applied studies with a proper basis and to make studies better comparable.

Given the different orientations of reviewed studies regarding the goals of participation, foci on different aspects vary accordingly. However, the focus of participation in general is mainly on how interactive PSS help facilitating the communication and collaboration and on issues related outcomes that can be achieved. While a few newer studies focus explicitly on the impacts of interactive PSS on aspects of learning [11,12,51], no studies have yet focused on analysing the social outcomes of PSS workshops. This matches with the observation of Goodspeed and Pelzer [34] that it is more likely that process attributes are measured, because outcomes are often hard to see, because they develop over a longer time period. Nevertheless, given the increasing need to engage with stakeholders under current planning regimes [76] to engage with stakeholders for the co-production of services [77], it is interesting for future studies to investigate what kind of social outcomes can be supported by means of interactive PSS applications. Therefore, PSS tools would have to be applied over a longer period, e.g., a series of workshops.

Last main finding worth to discuss here is that obviously scholars tend to report what works well, rather than what does not work. In all papers related to the different studies in total 176 positive quotes (including multiple quotes in one paper referring to the same criteria of participation) were found relating to one of the different aspects and criteria of participation, and only 20 negative statements (see Appendix B). It seems that authors are predominantly looking for evidences that prove their hypotheses and thereby devaluing negative findings, as it is rather unlikely that in only a bit more than 10 % of all instances, they did not succeed in doing so. This phenomenon known as the publication bias is reported by other scholars and occurring in various disciplines [78,79]. Fanelli [80] (p. 892) observes “various forms of conscious and unconscious biases that affect all stages of research […] producing positive findings when there should be none, thus creating distortions that are difficult to correct a posteriori”, particular “in fields where theories and methods are less clearly defined, and true replication is rare or impossible” Having tested 4600 papers from different disciplines over a period of 17 years (1990–2007) he further hypothesizes that research is becoming less pioneering and that the objectivity with which results are published is decreasing [80]. Nevertheless, as the exact number of positive or negative statements is not relevant for this study, because multiple quotes are counted as one per study (see Section 4), this observation does not significantly affect the reliability and relevance of our findings.

6.3. Reflection of the Methodology Applied in this Review

The methodology applied in this review includes various aspects that might limit the validity of the obtained results. First, the characteristics of the various interactive PSS presented in Appendix A, Table A1, Table A2 and Table A3 (applications, tools, processes) result from the interpretation of the authors if explicit details were not given. Moreover, when a ‘no’ is given in the tables it means it was not given in the paper, which does not necessarily mean that it was not done. It can of course be that the authors of the study simply did not explain it. Second, the hypothesized and measured impacts of the various studies on the different aspects and criteria of participation as analysed in Section 5.2 (see also Appendix B, Table A4, Table A5, Table A6 and Table A7) is an intersubjective endeavour, particularly when hypotheses are not explicitly spelled out by the authors. In order to minimize this subjectivity and to increase inter-rater reliability [81] all statements were cross-checked by the co-authors. Third, certain impacts of the use of interactive PSS in stakeholder workshops that were reported by the authors were not taken into account, due to the theory driven analysis method applied [16]. For instance, some studies reported that the use of the maptables created enthusiasm among participants [20,37]. Enthusiasm is for sure something the use of a maptable might create, but in our view, it does not necessarily lead to better or more participation. Therefore, it is also not included in any of the studies used here to develop the framework in Section 3. On the contrary, in our own studies we have observed enthusiasm rather limiting a serious and productive use of the interactive PSs, and therefore would have to be overcome in order to institutionalize such tools in everyday practice.

6.4. Future Research Needs

Findings from the review allow identifying the following future research needs in relation to interactive PSS. First, most tools reviewed here are developed using proprietary GIS software which is in most cases not explicitly developed for tangible user interfaces. Open source tools offering potentials to further refine applications by others or in future case studies are so far very much neglected. Future research studies should explore impacts of interactive PSS tools made explicitly for tangible user interfaces on their user friendliness and its correlation with the usefulness of the tool [14,82].

Second, only a minority of reviewed studies reported the inclusion of users and/or stakeholders in the PSS tool development process. A stronger integration of them in the development process is recommended by scholars as being integral for enhancing uptake and implementation of PSS [15,83]. Thus, future interactive PSS studies shall research the challenges as well as added values of including users and stakeholders in the tool development process, e.g., following a human centred design (HCD) approach [84].

Third, the review revealed that social outcomes of the use of interactive PSS in stakeholder workshops are hardly researched yet. Thus, in suitable long-term case studies potential social outcomes such as community building or increased ownership through interactive PSS workshops series might be investigated. In this context te Brömmelstroet [82] suggests to research how more flexible PSS that that can be quickly adapted and contextualized [85] allows the actors to develop a sense of ownership and adapt the PSS to their specific demands.

Finally, the adoption of results from interactive PSS workshops into ongoing planning processes needs to researched. To this end Geertman [6] recommends to study successful or best practice studies and to learn from them. Te Brömmelstroet ([83] p. 80) proposes a research programme that combines “context-rich and control-rich [PSS] research environments” in order to examine the uptake of PSS research in planning practice.

7. Conclusions

This review analysed how interactive PSS have contributed to the goal of strengthening stakeholder participation. Findings from the 16 studies reviewed here show that the implementation of maptable-based interactive PSS in planning related studies has increased significantly since the first case study in 2006 in numbers of tools developed, geographic regions covered, and in the variety of topics and tasks addressed. Over the years, not only the tools and applications have become more sophisticated and complex, also the goals of the studies using such tools have changed from involving stakeholders in designing interventions [20,55] to involving stakeholders in earlier planning stages, e.g., in order to develop a joint problem understanding or integrating knowledge [12,51]. In line with that, the interest of researchers in such studies has evolved from basically exploring the usability of interactive PSS for stakeholder participation [57] to more explicit research questions related to stakeholder participation such as the added value of interactive PSS on group-based learning [37,55].

Based on the second part of the analysis we can conclude that in all case studies the interactive PSS has contributed to strengthening participation, because all studies reported positive statements regarding the different aspect sand criteria of participation. Nevertheless, a key finding is that many case studies of interactive PSS reviewed here lack a proper framework and operationalization for investigating the effects of the tools and applications on the different aspects and criteria of participation. Though studies usually hypothesize certain impacts of their tools on participation and apply suitable methods for measuring and evaluating such impacts in PSS workshops, in many studies a certain mismatch was detected. Many studies are much more detailed in measuring the impacts on participation, e.g., by evidencing impacts on the transparency or inclusiveness of the public dialogue, than hypothesizing them before. Other studies provide detailed hypotheses of impacts but fall short in evidencing them by means of evaluation. Only three of the reviewed 16 studies, come up with a proper framework upfront and systematically measure impacts of those aspects of participation hypothesized upfront. In conclusion, further theoretical studies conceptualizing impacts of interactive PSS on participation are needed as well as more empirical studies, testing these impacts in real world case contexts with various groups of stakeholders.

Moreover, most studies analysed result from a more or less strict research context. Consequently, the adoption of the tools and applications in planning practice, e.g., through the involved case study partners, is not reported. Thus, no straight conclusions can be drawn from the review regarding whether or not the implementation of interactive PSS might have the potential to mitigate the often-stated PSS implementation gap. On the contrary, findings from the review help identifying issues that needs to be addressed in future studies in order to mitigate the implementation gap, such as the use of open source software explicitly made for maptables, the inclusion of users in the PSS development process, and the in depth analysis of results from PSS workshops in planning processes (see Section 6.4).