Street Experiments Across EU Cities: An Exploratory Study on Leveraging Data for Urban Mobility Impact Evaluation ^†

Abstract

European cities are under pressure to be at the forefront of climate neutrality while providing inclusive, safe, and sustainable urban mobility. Street experiments are being adopted to accelerate this transition, yet assessing their impact remains challenging. This study addresses this gap by providing an evidence-based impact assessment of street experiments. The research builds on insights from 20 European cities, including 13 from the EU Cities Mission, regarding expected goals and current evaluation barriers. A preliminary quasi-experimental spatial and temporal approach is proposed and further enriched through the identification of the most relevant mobility domains and indicators addressed by cities. An exploration of data collection technologies is undertaken to meet the cities’ needs, culminating in the design of a portable and easy-to-install laboratory, the Labkit, for in situ and non-intrusive evaluation of public space interventions. The Labkit is tested and validated in an open area with a constant flow of pedestrians, cyclists, e-scooters, and vehicles. The results of this testing process, along with feedback from cities regarding the methodological approach and potential indicators, are analysed. The study concludes with a discussion of the opportunities and limitations of data-driven approaches for urban mobility impact assessment and the proposal of future research directions.

1. Introduction

Sustainable mobility is essential to achieving climate neutrality, overcoming social inequalities [1], and fostering a competitive and resource-efficient transport system [2]. Beyond the efficient movement of people and goods, the discussion around transport has shifted towards addressing broader issues, such as global and local environmental effects and the wider social implications for health and inequality [3,4], among others. Urban planners and transport policymakers deal with complex urban dynamics that fluctuate from the vastness of regions to the bustling activities of streets. This requires a continuous evaluation, not only for the planning and implementation of mobility solutions but also for the subsequent impact assessment. Beneficial mobility interventions should be promoted and replicated based on evidence to increase public support [5]. Building on previous progress by the authors regarding the availability and pilot testing of market-ready data collection technologies for public space monitoring [6], this paper expands the insights into the impact evaluation of temporary pilot projects known as street experiments (SEs) [7].

On a global scale, cities must meet the challenges of the current climate crisis [8]. Transport is a key sector in achieving climate neutrality targets due to the difficulty of decoupling its contribution to economic development from its environmental impact [9].

In response to this challenge, the European Union (EU) Mission on Climate Neutral and Smart Cities, also known as the Cities Mission, was launched to support and promote 100 cities in their systemic transformation towards climate neutrality by 2030 [10]. In terms of decarbonisation, the Cities Mission applicants, representing 18% of the EU population, are mainly focusing on technological solutions, such as electrification, which may not be sufficient if mobility levels remain unchanged [11], if the externalities of electric vehicles are neglected [12], or if benefits are undermined by the so-called rebound effect [13,14].

Beyond its climate change contribution, urban transport generates various externalities that negatively impact mobility systems. The most critical ones are the invasion of public space for road construction, local air pollution, road accidents, congestion, vibration, and oil dependency, followed by barrier effects, noise, and visual blight [15]. Dependence on individual modes of transport has also led to sedentary lifestyles and a polluted environment, which affect people’s physical and mental health [16,17]. The promotion of active mobility and public transport appears as a synergistic strategy to reduce pollution and congestion, increase safety and accessibility, improve the mental well-being of citizens, and even generate savings for society. These benefits are seen as equally positive by people from different backgrounds [18]. Even if there is initial public resistance to traffic restrictions, adopting sustainable mobility approaches enables a positive economic feedback loop with local businesses [19].

In this sense, current approaches to street design focus on the ‘fair’ distribution of public space, generally seeking to rebalance space between motorised and non-motorised transport and even between transport and other uses of public space [20,21,22]. Cities increasingly serve as testing grounds for SEs, a distinct type of pilot project that temporarily alters the use, regulation, design, or function of specific street sections, entire streets, or even larger urban areas. SEs offer a faster and cheaper alternative to permanent structural changes for the introduction of new space-distribution models [7]. Some examples of SEs are tactical urbanism interventions for intersections and street reconfigurations [23,24]; repurposing of street parking spaces with ‘parklets’ [25]; the repurposing of entire city streets through the approach of ‘play streets’ [26,27], ‘open streets’ [28], and ‘ciclovías’ [29]; and even new district-planning approaches such as car-free superblocks [30,31], district pedestrianisation, and low emission zones [5]; and the 15 min city concept [32,33].

Despite the growing adoption of SEs, particularly accelerated by the demand for expanded public space during the COVID-19 pandemic [34,35], a critical knowledge gap remains. Specifically, there is limited understanding of both local and citywide impacts, as well as of the contextual factors influencing SE effectiveness [36,37]. There is a significant lack of available methods for planners and policymakers to make evidence-based decisions [15] despite the potential of data-driven decision-making to enhance system efficiency, governance, and sustainability [38]. Since urban mobility is becoming increasingly complex, traditional methods may fail to capture the intricate interactions within urban spaces [39] and to address the diverse needs of pedestrians, cyclists, and users of new mobility services such as e-scooters. International indicator frameworks operate at an urban scale, overlooking the scale limitations of local SEs at the street or block level [40,41,42].

In this context, the paper aims to contribute to closing the gap between the implementation of SEs and the evidence of subsequent impacts by addressing the following research questions (RQs):

(i)

RQ1. What limitations do the experimental, temporal, and spatial scope of SEs impose on impact evaluation approaches?

(ii)

RQ2. What is the expected impact of adopting SEs from the perspective of EU cities?

(iii)

RQ3. How might outdoor data collection technologies support the impact measurement by maintaining the experimental flexibility of SEs?

The structure of the paper is as follows: Section 2 describes the methodological steps followed to address the research questions. Section 3 presents the results from the literature review on evaluation frameworks to assess the impact of SEs and the insights gathered from cities regarding SE goals. This section also includes the proposal for an innovative approach, called the Labkit, to facilitate an in situ and non-intrusive evaluation of SEs. Section 4 provides a critical discussion on the barriers and challenges associated with assessing the success of SEs, together with the opportunities and limitations of the Labkit approach. Finally, Section 5 presents the paper’s conclusions and outlines future research directions.

2. Materials and Methods

The methodological steps to address the research questions are based on several activities that allow, firstly, the proposal of an approach to evaluate the success of SEs, considering the limited spatial reach of their impact and the temporal entanglement with wider city trends. Secondly, the identification of expected impacts from cities implementing SEs and the most relevant indicators to measure achievements. Thirdly, data collection technologies are reviewed, and market-ready solutions are explored to collect the variables required. Finally, insights are integrated into an early implementation of the Labkit in an open public space to test its functionalities and inform the design of data pipelines based on real-world data. These steps are further explained below, and a summary is represented in Figure 1.

2.1. Street Experiments Impact Evaluation Scope

The methodological steps begin with a literature review to establish the scope of an impact evaluation approach tailored to SEs. This review examines urban mobility challenges and opportunities, standardised indicator frameworks, and previous studies on similar evaluation approaches.

In response to the urban mobility challenges outlined in the introduction, SEs emerge as an attractive alternative due to their rapid and flexible approach to testing potential solutions. However, their success depends on multiple factors that must be carefully considered during the impact evaluation to ensure meaningful and context-sensitive results. According to the assessment framework on SEs proposed by Kinigadner et al. [36], the impacts can be categorised into two contexts: system context and experiment context. The system context includes long-term, indirect changes related to policy, financial, and regulatory frameworks, as well as shifts in mindset, norms, and stakeholder networks. For instance, citywide policies may be influenced by SE implementations, leading to the introduction of new regulatory measures (e.g., speed limits), market-based instruments (e.g., taxes), or information-based strategies (e.g., awareness campaigns) [43]. Such policy implications may also be crucial to increasing the adoption and use of active and shared mobility services such as electric-bike-sharing systems [44].

Experiment context impacts include transport-related and sustainability-related changes, which tend to be more immediate, direct, and locally visible than system context impacts. These effects can be monitored before, during, and after the implementation of an SE. Experiment context impacts can be either positive or negative, for instance, changes in public perception towards the acceptance or rejection of a solution, or actual reductions in traffic volumes and safety incidents, or merely their displacement [36].

Despite the reported evidence regarding SEs, there is still a gap in comprehensively understanding the direct effects of measures such as quick street-space reallocation [45,46], tactical urbanism [47], or superblocks [48,49]. Among the types of studies evaluating SE impacts, ex-ante evaluations based on modelling approaches are common [50,51], while other studies assess people’s perceptions and choices at the case study level via surveys and observations [5,52]. The air quality and health impacts of SEs tend to rely on environmental monitoring and health surveys for before-and-after comparisons [49].

Another more complex approach applied in transport is the use of quasi-experimental methods to capture the effects of mobility solutions that overlap with other interventions or trends that are likely to generate causal spatial spillover effects [53]. The difference-in-differences (DID) approach, for instance, identifies the effect of a mobility solution by first calculating the change in outcomes over time for both an intervention area and a control area. By differencing these time-based changes, the DID approach removes unobservable factors specific to each area, as well as any shared temporal trend [54]. This approach is also aligned with the quantification of before, business-as-usual (BAU), and after scenarios, as recommended for the evaluation of urban mobility measures [55].

In this regard, the monitoring of changes over time is usually carried out through the definition of key performance indicators (KPIs). Although there is no existing indicator framework to measure such changes at the scale required by local SEs [56], the EU relies on two standards to assess their mobility systems: the Sustainable Urban Mobility Indicators (SUMIs) [57] and the CIVITAS Evaluation Framework [58]. These allow for the identification of urban mobility strengths and weaknesses at the city level, but their main limitation lies in the significant resource requirements and the data management needed to support extensive data collection and complex calculations [59].

Based on this review, the scope of the impact evaluation framework for SEs is limited to the following: (i) focusing solely on experiment context impacts, which relate to direct, local changes in sustainability and transport; (ii) adopting a DID approach to track changes over time and space; and (iii) leveraging EU indicator frameworks to facilitate debate among the participants cities.

2.2. City Insights: Theory of Change Workshops

To identify the most relevant urban mobility goals and indicators for cities implementing SEs, two workshops were conducted. The first one was organised with 12 cities participating in the European Living Lab on Designing Sustainable Urban Mobility Towards Climate Neutral Cities (ELABORATOR) project: Copenhagen (DK), Helsinki (FI), Milan (IT), Zaragoza (ES), Issy-les-Moulineaux (FR), Trikala (GR), Lund (SE), Liberec (CZ), Velenje (SI), Split (HR), Krusevac (RS), and Ioannina (GR) [60]. Additionally, a second group of eight cities was involved, with the participation of representatives from Amsterdam (NL), Riga (LV), Vilnius (LT), Kozani (GR), Braga (PT), London (UK), Cugir (RO), and Vratsa (BG). In total, 20 European cities, including 13 from the Cities Mission, were involved during this stage.

The workshops were designed using a theory of change (ToC) approach to enable the definition of plausible pathways linking SEs to expected outputs and outcomes, allowing participants to articulate the theories that will drive change [61,62]. The ToC workshop consists of three sequential rounds, with a total duration of one and a half hours. Participants were divided by cities, with each city group composed of city representatives and technicians along with supporting technical partners (e.g., universities, technology centres, etc.). In the first round, cities worked individually, whereas in rounds two and three, cities were paired based on the similarity of their SEs. This collaborative set-up was intended to facilitate the exchange of ideas and best practices, enabling cities to learn from each other’s experiences and insights.

During the first round, each city brainstormed expected impacts and goals by first envisioning the problem to be solved and the planned SEs. Participants were encouraged to consider how their projects will address current urban challenges and contribute to the city’s future vision. Both impacts and goals were visually mapped on the impact canvas (see Figure 2a) and categorised into short-, medium-, and long-term impacts or end goals of the SE. This visual aid served as a communication tool within the workshop and as a valuable reference for further planning and development.

During the second round, cities were asked to complete the first half of the impact evaluation card (see Figure 2b). In this round, participants were required to state each impact from the canvas and select an indicator that could be used to measure it. To facilitate indicator selection, a list describing SUMIs and CIVITAS indicators was provided. This also served to limit the range of options and to ensure comparability across the roundtables. After selecting an indicator, cities explained how it is relevant for quantifying the corresponding impact.

In the final round, based on the selected impact and indicator, cities completed the impact evaluation card by identifying of the most appropriate data collection method. Participants were encouraged to consider the logistical and technical requirements of their data collection strategies in order to move forward with the evaluation of SEs. The cards served as the main output of the workshop, and all the insights from the cities were then processed to identify the most relevant impacts and indicators based on their frequency of selection.

2.3. Technologies Supporting the Impact Evaluation of Street Experiments

During the third methodological step, a review of on-street data collection technologies was performed to initiate the scouting of market-ready devices. The challenges and opportunities of new technological approaches were addressed in two reviews: one on intelligent transport systems (ITSs), covering the literature from 2006 and 2014 [63], and another on smart mobility technology trends from 2011 and 2020 [64]. Although there remains a gap in standardised quantitative frameworks, the main uses of such technologies in urban mobility include (i) continuous data collection for monitoring and management; (ii) smart surveillance for road safety; and (iii) monitoring traffic conditions and real-time responses to emerging situations. Among the technologies enabling such applications, the use of sensors and the internet of things (IoT) are at the forefront of real-time data collection. Other key technologies along the data pipeline include (i) artificial intelligence (AI), (ii) geospatial technologies, and (iii) big data. The latter serves as the foundation layer, processing vast amounts of information from multiple sources to generate actionable insights for mobility planning.

The range of devices for street characterisation varies across several types of sensor technologies and their combinations. The following examples illustrate the diversity of available tools. Personal wearable trackers, composed of GPS receivers and accelerometers, have been tested to monitor walking behaviour and acquire continuous, fine-grained tracks [65,66]. Media access control (MAC) detection via Wi-Fi and Bluetooth probes have been used to analyse visitor trajectories and volumes over time, in transport stations [67], and gated communities [68], and for monitoring pedestrians and cyclists [69]. Infrared counters have been applied to estimate pedestrian presence, movement, and patterns over time and space, for example, in a ten-block urban festival setting [66]. Long-range wide-area networks (LoRaWANs) have been installed in public squares to quantify environmental indicators and the use of public furniture [70]. Smart cameras with image processing capabilities have been employed to monitor social distancing in public spaces and to count vehicles, cyclists, and pedestrians in busy streets [71]. Finally, light detection and ranging (LiDAR) have been applied to 3D modelling of streets [72], as well as to pedestrian and safety monitoring [73].

Despite the wide availability of data collection technologies, an effective impact evaluation framework for SEs requires alignment between methods and the specific context and priorities of cities. This is the aim of the final methodological step, as detailed in Section 2.4.

2.4. Labkit Concept and Design

Finally, this exploratory study contributes to filling the knowledge gap in the impact evaluation of SEs by designing and testing a portable urban mobility laboratory, called the Labkit. As a concept, the Labkit represents an innovative approach that aligns with the characteristics of SEs by enabling in situ, rapid, and non-intrusive measurement of the variables and calculation of indicators required by the cities. Its design integrates the findings from the previous methodological steps, particularly the results of the ToC workshop, into a practical and actionable tool to support cities implementing SEs.

From an operational point of view, the Labkit components must meet the following constraints. First, market-readiness; as the participating cities will be implementing SEs in the short term, there is a need for accurate and reliable technologies. Second, portability, as the Labkit is intended to be an easy-to-install, rapid and flexible solution aligned with the experimental scope of SEs. However, since the Labkit may also support the design of the monitoring layer, the selected technologies should also have the potential to become permanent. Third, non-intrusiveness, due to the sensitivity of data privacy in open urban spaces. Finally, the devices should perform adequately in outdoor conditions, and the technologies should operate effectively at normal urban speeds.

The data collection and analysis pipeline of the Labkit is designed based on a preliminary test in an open urban environment. The chosen area is a pedestrianised street in Zaragoza characterised by constant movement of pedestrians, cyclists, e-scooters, and delivery vehicles. The pilot aims to understand the advantages and limitations of the different types of data collection technologies tested and to inform the design of the Labkit’s data pipeline. This exercise does not draw conclusions about the mobility conditions of the monitored area. On the contrary, the site is selected based on the authors’ understanding of local mobility trends, allowing the reliability and functionality of the Labkit to be tested.

Once deployed, the Labkit begins the data collection phase. In addition to vehicle, cyclist, and pedestrian counts and speed monitoring, this phase also captures noise levels, air quality parameters, as well as weather conditions without disturbing the urban landscape. The subsequent analysis of the collected data applies analytical methods to transform raw data inputs into actionable insights. This process follows the three steps of a data-driven project applied in transport studies: data collection, data preparation, and data modelling [74]. As presented in Section 3, this process culminates in the calculation of key urban mobility indicators. The limitations encountered during the testing and validation of the Labkit are discussed in Section 4.

3. Results

3.1. Impact Evaluation Framework for Street Experiments

In the context of transport-related studies, it is not advisable to assume that a behavioural change perceived locally is directly linked to the SE implemented. Transport systems are, by design, networks of interconnected components that interact with the urban environment in various ways. For example, a road safety intervention that results in improvements in one location may lead to the migration of crashes to nearby streets due to the displacement of traffic flows. The effects of a transport intervention may spill over to locations outside the immediate area of influence of an SE, or vice versa [53].

For this reason, the proposed approach to evaluating the impact achieved by an SE follows the quasi-experimental conditions of the DID method. To this end, the following areas are defined: (i) the SE area, where the solution takes place, including the existing transport infrastructure and services, and people travelling through it; (ii) the control area, a comparable space in structure and behaviour that should minimise differences with the SE area to allow comparison; (iii) the surrounding streets of the SE area, where traffic might be diverted, suggesting that any perceived reduction in externalities within the SE area may result from displacement rather than an actual decrease.

As shown in Figure 3, this spatial framework enables a dual evaluation, encompassing the before-and-after comparison within the SE area whilst also incorporating the BAU scenario, as reflected in the trends observed in the control area. This approach considers urban, regional, or global trends (i.e., impact of other factors) that may contribute to the perceived success of the SE. However, these external trends should be subtracted from the difference between the pre-solution (before) and post-solution (after) values to determine the actual impact of the SE (i.e., the changes introduced by the solution).

3.2. Indicator Identification and Selection by Cities

The first step in identifying potential indicators in line with the cities’ objectives is a review of the literature. SUMIs are categorised based on the type of data sources and different types of mobility solutions they address [57]. Additional indicators are included to complement the existing list of SUMIs, and their categorisation is based on similarity with other indicators and the authors’ expertise.

Table 1. Indicators selected by cities and related types of interventions.

SUMIs	Indicator Category	Data Collection Category	Type of Intervention Addressed by Indicators [57]						Frequency of Selection by Cities During the ToC Workshops
			TOD	TP	CFP	BL	WS	BS
Quality of public space	QL	S	*		*	*	*	*	14: LUN, COP; TRI, HEL, ZGZ; VAL, ISSY, SPL, AMS, MIL, BRA, LND, RIG, VIL
Accidents	QL	DB	✕	✕	✕	✕			11: LUN, COP, TRI, HEL, VAL, KRU, LIB, MIL, BRA, LND, VIL
Traffic safety active modes	QL	DB	✕	✕	✕	✕			10: LUN, COP, TRI, HEL, VAL, ISSY, SPL, BRA, LND, VIL
Opportunity for active mobility	MSP	GIS	✕	✕	✕	✕	✕	✕	10: LUN, COP, ZGZ, VAL, MIL, AMS, BRA, LND, RIG, VIL
Urban functional diversity	MSP	GIS	*		*		*		8: LUN, COP, ZGZ, HEL, AMS, BRA, LND, RIG
Satisfaction with transport	MSP	S	✕						7: LUN, TRI, ZGZ, MIL, KRU, LND, RIG
Air pollutant emissions	QL	DB, S, TM	✕	✕	✕	✕	✕	✕	6: KRU, LUN, ZGZ, BRA, LND, VIL
Noise hindrance	QL	F	✕	✕	✕				6: ION, ZGZ, SPLI, BRA, LND, VIL
Access to mobility services	QL	GIS	✕		✕				5: TRI, ZGZ, MIL, SPLIT, RIG
Congestion and delays	EC	S, GIS, F	✕	✕	✕	✕	✕	✕	5: LUND, ZGZ, VAL, SPLIT, BRA
Accessibility for mobility-impaired groups	MSP	S	✕				✕		5: LUN, TRI, MIL, LND, VIL
Security	MSP	S							5: LUN, ION, MIL, RIG, VIL
Mobility space usage	QL	GIS	*	*	*	*	*	*	5: AMS, MIL, BRA, LND, RIG,
Greenhouse gas Emissions	ENV	DB, S, TM	✕	✕	✕	✕	✕	✕	4: ION, ZGZ, BTA, LND
Modal split	MSP	DB, S	✕	✕	✕	✕	✕	✕	3: COP, ZGZ, KRU
Commuting travel time	EC	S	*	*	*	*	*	*	3: TRI, ZGZ, BRA
Multimodal integration	MSP	S, GIS	✕					✕	3: LUN, MIL, RIG
Affordability for the poorest groups	MSP	DB	✕		✕				-
Energy efficiency	ENV	DB, S, TM	✕		✕	✕	✕	✕	-

* Added by the authors. TOD: transit-oriented development; TP: traffic pacification; CFP: car-free planning; BL: bike lane; WS: walkable space; BS: public bike-sharing. QL: quality of life; MSP: mobility system performance; ENV: global environment; EC: economic success. S: survey; DB: based on existing databases; GIS: based on GIS; TM: traffic model; F: field observation. AMS: Amsterdam; BRA: Braga; COP: Copenhagen; HEL: Helsinki; ION: Ioannina; ISSY: Issy-les-Moulineaux; KRU: Krusevac; LIB: Liberec; LND: London; LUN: Lund; MIL: Milan; RIG: Riga; TRI: Trikala; SPL: Split; VAL: Velenje; VIL: Vilnius; ZGZ: Zaragoza.

presents, on the one hand, the most appropriate indicators regarding their frequency of use across six types of interventions: transit-oriented development (TOD), street calming or traffic pacification, car-free planning, creation of cycle lanes, walkable spaces, and implementation of public bike-sharing systems. On the other hand, Table 1 also summarises the indicators selected by cities through the impact evaluation cards completed during the ToC workshop. The final column specifies the frequency with which each indicator was selected and the cities that expressed interest in them.

Among the indicators selected by at least three cities, three groups are defined according to the category of data collection. The first group consists of indicators based on surveys, used to understand users’ level of awareness and perception towards the quality, satisfaction, and accessibility of the transport system. These indicators are considered valuable by cities aiming to design a better and safer distribution of public space informed by citizens’ opinions. The second group comprises geographic information system (GIS)-based indicators related to the availability and distribution of infrastructure, services, commerce, and facilities that promote active mobility. Cities identify these as fundamental for quantifying the quality of public space. The final group includes indicators derived from urban statistics and databases, covering topics such as safety and accidents, congestion and modal split, as well as environmental indicators such as air pollution and noise. This last group of indicators could be the most appropriate to be quantified using data collection technologies.

Cities identify the improvement in safety and the quality of urban space as their main objective for adopting SEs, with a focus on the most vulnerable users. There is also growing attention to emerging transport modes that increase pedestrian risk, such as e-scooters and other personal mobility vehicles (PMVs). The assessment of road safety impacts usually relies on crash and fatality statistics, which are published every one or two years, a periodicity that does not align with the short-term nature of SEs.

Cities also expect to have a positive impact on reducing car use and promoting active modes, as well as on reducing the environmental externalities of transport. The main obstacle discussed by the cities is how to effectively measure the impact of conventional city-level indicators, given that SEs are implemented at local scale. A third group of indicators is related to social aspects such as satisfaction, accessibility, and perception towards the transport system. Among topics not covered by the SUMIs, some cities emphasise the importance of considering climate adaptation, nature-based solutions, and spatial justice as trending design criteria.

3.3. Labkit Design and Testing

As previously commented, the Labkit aims to be an in situ, rapid and non-intrusive approach to collect variables and calculate indicators, in line with the flexible nature of SEs. From the exercise with cities summarised in Table 1, the indicators most suitable to be addressed by the Labkit are those that rely on databases and field measurements. Namely, accidents and the safety of active modes, noise, and emissions and congestion and modal split. From this point, the design of the Labkit starts with the matching between the variables to measure the capabilities of data collection technologies.

On the variable side, the indicators for each variable are initially assessed based on definitions from the SUMIs [57,59]. Both accidents and the safety of active modes focus on the number of fatalities caused by road accidents. This approach is slow-paced, as safety statistics are typically published annually or biennially. Noise is calculated as the population exposed to harmful levels, requiring direct field measurements. GHG and air pollutant emissions are calculated using activity factors (i.e., the distance travelled per transport mode and vehicle type) and emission factors (i.e., the quantity of pollutants emitted per unit of energy consumed). This method is suitable for calculating the direct effects of SEs if traffic volume monitoring is measured in the streets within and surrounding the SEs and control areas. Traffic volumes (e.g., cars per hour) are also crucial for assessing modal split and congestion, along with speed and direction. Since SEs typically aim to promote active mobility, pedestrian and cyclist volumes are critical variables. Lastly, based on feedback from the cities, thermal comfort is also considered, as it depends on meteorological data such as air temperature, specific humidity, wind velocity, and mean radiant temperature [75].

On the technology side, several types of devices have been identified to collect the variables mentioned above. These technologies meet the scouting criteria outlined in Section 2: market readiness, portability, non-intrusiveness, and outdoor functionality. In summary, Figure 4 illustrates the alignment between the available device types, variables, and indicators.

As a result, the Labkit is structured around the acquisition of commercially available devices, including pneumatic tubes for traffic and cycling lanes, radars for traffic and pedestrian paths, infrared counters for pedestrians, smart cameras for multimodal counting, and an air quality station equipped with electrochemical sensors for air pollutants and weather conditions. To understand the possibilities and limitations of the technologies aggregated in the Labkit, an early test is conducted on a pedestrian-priority street with low motor traffic and high volumes of bicycles and e-scooters in Zaragoza. As the test does not aim to evaluate the local mobility patterns, no numerical results are presented in order to avoid any misinterpretation of the mobility trends of the test area.

Nevertheless, data are collected to accumulate raw data from each device and to process and transform them into variables and then into the selected indicators. The test is successful for the following indicators: modal split, air pollutants, noise and thermal comfort. For these indicators, all required inputs are correctly measured, and the values are calculated. For congestion, although speeds are recorded for different times of the day, there is no distinction between peak and off-peak periods, as this is a low-speed, low-volume road. This also raises the question of whether congestion as defined by SUMIs is appropriate for a safe and quiet urban environment. In the case of GHG emissions, tailpipe emissions are not measured, this indicator is measured indirectly as CO₂ concentrations. Further testing is needed to determine whether this approach is appropriate. Lastly, safety variables are not collected because the tested smart camera device processes low-resolution images using edge computing. This enables vehicle and pedestrian counting but does not capture the detailed interactions or conflicts needed for safety assessment. These indicators will be considered in a future iteration of the technology scouting. As shown in Figure 5, the results are translated into a visualisation tool as an example of the monitoring potential of the Labkit approach.

4. Discussion

As debated with cities during the workshops, there is a need for an adequate impact evaluation approach capable of capturing the time- and space-constrained changes generated by SEs. For example, at the ToC workshop, cities questioned how citywide issues, such as GHG emission reductions or congestion decreases, could be effectively addressed given the limited scope of interventions focused on active mobility and improving the safety of pedestrians and vulnerable road users. This mismatch between the scale of impact evaluation and the scale of the SEs can lead to misaligned policies and missed opportunities for targeted improvements. In this sense, the adoption of quasi-experimental approaches might shed light on how effectively SE implementations translate into concrete changes in people’s behaviour or into the reduction of transport externalities.

However, cities also highlight concerns about resource constraints—limited budgets, time, or personnel—when attempting to measure mobility simultaneously in both the SE and a control area. Cities require methodological approaches that reduce this burden in terms of the number of externalities to address. SUMIs appear to consider this by including a set of 20 indicators, whereas CIVITAS, although flexible in terms of what cities can choose or discard, proposes more than 50 indicators. Working side by side with cities, as done in the ToC workshops, allows for the identification of the main issues that cities are focusing on. Whether in the form of expected impact, defined goals, or problems to tackle, these activities are useful for prioritising the mobility subjects that a set of indicators should include. Based on the insights gathered, the list of SUMIs could be potentially reduced by half.

In addition, cities are raising awareness towards other issues not covered by these indicator frameworks but which are becoming increasingly relevant when discussing new configurations of urban spaces, particularly in the case of SEs being co-created with citizens. Some participating cities express the need to consider climate adaptation and nature-based solutions, while others aim to go beyond accessibility and assess emerging social concepts such as spatial justice.

Although this study does not fully resolve the issue, the Labkit’s design contributes to bridging the gap between macro- and micro-scale evaluation methods, reducing the effort required to address traditional indicators. This approach not only enriches the evaluation and validation of SEs but could also support the definition of concrete lessons learned for policymaking and the promotion of sustainable mobility.

The preliminary evaluation of the Labkit has yielded positive results in terms of identifying the tool’s potential to define opportunities and constraints. While key indicators, such as air quality and modal split, can be quantified at the scale of SEs, congestion levels, traditionally based on vehicle speed, present a notable challenge. High speeds can be detrimental to pedestrian safety and discourage walking and cycling. This paradox highlights the need for a revised approach to assessing congestion that considers not only the speed of traffic but also the quality of life and safety of urban spaces. Similarly, reliable and privacy-protecting technologies can help addressing the lack of monitoring of safety and conflicts between modes. Technologies, such as AI cameras and LiDAR, could be tested to assess their potential to analyse traffic patterns and pedestrian behaviour.

Another key benefit observed during these early tests is the Labkit’s ability to provide rapid, on-site analysis, a feature that could address some of the resource limitations expressed by cities. Collecting essential mobility information in such a flexible way might be critical for monitoring the SE and control areas, making timely decisions, or encouraging citizen participation. The Labkit’s portability and non-intrusive nature ensure that its deployment causes minimal disruption to the existing urban fabric. It can also support the design of the city’s data collection infrastructure and the early testing of monitoring and visualisation interfaces.

However, early deployments have also highlighted challenges, mainly regarding the emerging environmental and social issues raised by cities. Further adjustments and complementary activities are needed to adapt the tool more closely with cities’ expectations. This approach could extend its applicability to diverse urban environments, ultimately contributing to the development of smarter and more sustainable cities.

5. Conclusions

Regarding RQ1, the main limitation recognised by cities is the difficulty in attributing concrete impacts to local SEs on phenomena that occur at the city or regional level, such as pollution or car dependency. While existing evaluation frameworks provide a foundation, there is a clear need for methodologies tailored to the temporal and adaptive nature of SEs. This is also related to cities’ concern towards the resource demand of existing frameworks and the limited budget, time, or personnel that city administrations might allocate to generate solid evidence. On one hand, the need to quantify the real effect of SEs is recognised, requiring the generation of before-and-after and BAU scenarios. On the other hand, this quasi-experimental approach depends on the selection and monitoring of at least two areas, the intervention area and an additional control area, doubling the required effort. All these limitations need to be addressed to assist participating cities in the impact evaluation of SEs and for the generation of evidence-based best practices and policy recommendations.

In the case of RQ2, the ToC workshops resulted in the identification of the expected impacts and goals cities are pursuing through the deployment of SEs, as well as of the mobility fields they aim to address through indicators. The quality of urban space and safety are at the top priorities for implementing SEs that can reallocate car space, reduce speeds, and open streets to people. Encouraging behaviour change towards active mobility and achieving associated local environmental benefits are also key objectives for cities. At a third level, cities highlight concerns related to the quality of the transport system and the perception of citizens. While these goals are directly linked to the experimental context impacts, they are also connected to the system context impacts of SEs, which can generate long-term effects through changes in policies or mindsets [36]. The implementation of SEs might be an opportunity to merge engineering approaches with social sciences approaches to consolidate sustainable mobility.

From this exercise, two key challenges emerge. First, addressing other topics not covered by EU frameworks [58,59] such as climate adaptation and spatial justice—concepts that might be difficult to translate into measurable indicators. Second, evaluating existing indicators to determine whether they are appropriate for assessing the goals of SEs. For instance, current practices for measuring safety and congestion may not fully reflect the impacts targeted by these interventions.

Finally, insights about RQ3 are based on the design and pilot trial of the Labkit. The Labkit has demonstrated significant potential in initial field trials to support how cities approach the measurement and analysis of SEs. Although the Labkit currently addresses only a subset of the identified indicators, its ability to conduct non-intrusive, in situ evaluations enables the agile adaptation of public spaces. Future research should focus on integrating GIS- and survey-based methods to provide a more comprehensive data collection framework. Such integration would enhance the Labkit’s outputs by encompassing a broader range of indicators.

Additionally, future research should include continued collaboration with cities to develop a tailored evaluation methodology that integrates the quasi-experimental approach with city expectations and constraints. This would be a valuable development to provide evidence-based insights into the transformations driven by SEs and to help cities identify best practices and replication opportunities. However, this will require addressing several challenges, such as minimising resource demands considering cities’ budget and personnel constraints. Further concerns include potential inconsistencies and issues of scalability, which may affect the comparability of results and the harmonization of approaches across EU cities. The reliance on local data also presents challenges, particularly if the data are incomplete or of poor quality. To begin tackling these challenges, a key next step would be to conduct a comprehensive literature review of indicators that are better suited to the nature of SEs, while also incorporating emerging concepts such as mobility justice, nature-based solutions, and climate adaptation.