Next Article in Journal
Augmenting Advanced Analytics into Enterprise Systems: A Focus on Post-Implementation Activities
Next Article in Special Issue
Sustainability Assessment of Asset Management Decisions for Wastewater Infrastructure Systems—Implementation of a System Dynamics Model
Previous Article in Journal / Special Issue
Addressing the Sustainability Issue in Smart Cities: A Comprehensive Model for Evaluating the Impacts of Electric Vehicle Diffusion
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Measuring the Change Towards More Sustainable Mobility: MUV Impact Evaluation Approach

LINKS Foundation (Leading Innovation & Knowledge for Society), via Pier Carlo Boggio, 61, 10138 Turin, Italy
Department of Architecture, Design and Media Technology, Aalborg University, 9000 Aalborg, Denmark
Department of Planning, Aalborg University, 9000 Aalborg, Denmark
PUSH Design Lab, 90133 Palermo, Italy
Author to whom correspondence should be addressed.
Systems 2019, 7(2), 30;
Submission received: 30 April 2019 / Revised: 5 June 2019 / Accepted: 18 June 2019 / Published: 20 June 2019


Urban areas can be considered the ground for the challenges related to the UN’s Sustainable Development Goals (SDGs). The objective of shaping cities as human settlement that will see a more inclusive, safe, resilient, and sustainable future is often argued in literature as an issue dependent on behavioral change of inhabitants in urban areas. In this paper, the authors question if experimental applications based on gamification can co-produce more sustainable neighborhoods through an impact evaluation method that departs from individual choices within the complex of urban mobility. This investigation is carried out within MUV (Mobility Urban Values), an EU research and innovation project, which aims to trigger more sustainable urban mobility in six pilot cities. This article describes the critical method of validation, an impact assessment of the MUV experimental gamification in the pilot cities, in order to represent a proof for future urban strategies. This methodological approach is based on an evaluation structured on indicators of both impact and process suitable for urban contexts. As based on six pilot cities, with possibilities for transferability to other contexts and scalability to other cities, the method represents a reference work for the evaluation of similar experimental applications.

1. Introduction

In recent years, urban sustainability has become a topic with special attention on policy and research. This is manifested also by the United Nations Sustainable Development Goals (SDGs) [1], which dedicate an entire objective to cities in the 2030 Agenda for Sustainable Development specifically to ‘make cities and human settlement inclusive, safe, resilient, and sustainable’ (goal 11). Urban areas are touched in one way or another by almost all the UN’s SDGs, with the need of addressing also issues, such as road traffic (e.g., goal 3.6) and air pollution in cities (e.g., goals 3.9 and 13.2). In parallel, researchers, private companies, and public bodies are nowadays focusing on sustainability approaches that will turn to participative and people-oriented solutions in future cities. Approaches of this kind include questions on how people can assume positive attitudes and routines regarding the use of non-renewable resources. Moreover, the literature on sustainable change often focuses on how people can individually be oriented and sometimes persuaded to use fewer resources; for example, raising awareness on the benefit to achieve more systemic and collective sustainable transitions. Such a movement seems to take as a basic principle what Stephen Wendel pointed out [2]: people are reluctant in changing their habits. Individuals rarely recognize their power to affect and improve the liveability of the context they inhabit as their own neighborhood/city/region/planet and as their own life. It is difficult, indeed, to believe in the ‘butterfly effect’ (i.e., our individual routines have an impact on an entire ecosystem), so that the causal link between every little change of daily routines can have an impact on a systemic dimension of the SDG as fundamental and relevant.
With the attempt to enhance this awareness, studies on behavior change highlight gamification as an effective perspective [3]. Gamification is often defined as the process of game design elements that structure playful activities [4] on non-game contexts [5]. Game design elements are generally driven by a user-perspective that leads individual human personal motivation and/or perception to provide more effective, efficient, engaging, enduring, and entertaining experiences [6,7]. Points, ranks, levels, competitions, challenges, rewards, badges, or reputations are designed to keep users, as players, in the game [8]. Gamification has been much used as a tool for user-design products or services, but also in business with the purpose to motivate individual behavior change.
Since the beginning of the XX century, psychologists, anthropologists, and philosophers have studied the function of playing for human beings. Karl Groos in his “The play of man” [9], besides acknowledging that the instinct of playing has deep physiological bases, also argued that “play” has a fundamentally social function: it offers to humans (and animals) a tool to mastery those activities that bring prosperity for their species. Later in time, Bernard Suits [10] argued that when individuals engage in gamification, they change the temporary perception of their individual experience sometimes breaking habits: “playing a game is the voluntary attempt to overcome unnecessary obstacles”. Further studies on gamification have pointed to diverse psychological motions (e.g., social motivation, intrinsic benefits, monetary, and/or personal rewards), which have proven to have the power to change people choices about their own behavior as, for example, Mihaly Csikszentmihalyi’s [11] study. Gamification can be seen as a means to facilitate change at the level of individual unsustainable habits by promoting more informed, enjoyable, environmental, and social friendly choices.
In the last decade, all over the world, physical and virtual urban games are being designed trying to demonstrate these hypotheses, but how can the gamification of urban mobility experiences become effective and valuable?
MUV (Mobility Urban Values) is a research and innovation action (2017–2020) based on an experimental approach to gamification applied in six European neighborhoods (Buitenveldert in Amsterdam, Sant Andreu in Barcelona, the historic district of the Portuguese county of Fundao, Muide-Meulestede in the harbour of Ghent, the new area of Jätkäsaari in Helsinki, and the Centro Storico in Palermo). The MUV project [12] is a concrete critical attempt to experiment with a gamification approach, with the main objective to engage people to promote a shift towards more sustainable and healthy urban mobility. MUV engagement strategy of co-creation and co-design [13] is based on urban governance and participatory design theories [14,15] that aim at triple-loop learning [16]. Public involvement of citizens and policy actors occurs progressively through thin and thick levels of participation [17] and serves in the co-creation of game communities and in the co-design of game solutions that aim to enhance interaction and to transform urban policy through a ‘conversational planning’ among a variety of actors (from communities to public authorities and vice versa).
MUV method of co-creation and co-design develops socially through a direct dialogue with the local communities and stakeholders, and technically through a mobile app, a network of air monitoring stations and dashboards, all designed through participatory and collaborative methods.
The effort to raise individual and collective awareness on car-dependency and air pollution is aimed to provide an enjoyable experience of the game for citizens with returns on an evidence-based approach to urban policies inspired by people-centered mobility data collected. The active involvement of citizens and other local stakeholders can consequently result in the long term, more efficient, and cost-effective urban planning processes, while achieving global sustainability goals.
The MUV app enables an activity-based game, previously discussed in [18], through a metaphor of sporty narrative that connects diverse kind of users: (i) citizens as MUVers play as athletes to get rewards for their sustainable mobility choices, i.e., walking, biking, and using public transportation, (ii) public authorities as trainers provide training sessions to coach athletes to improve their sustainable mobility skills, and (iii) local business communities, as sponsors, have the opportunity to promote their brand and their products through the athletes’ best achievements. The MUV app (Figure 1) connects all these users when MUVers press a start button choosing their sustainable transport mode at the beginning of their journey, and press it again when they arrive at the destination. As a result of playing MUV, spatial-temporal mobility data of inestimable value are collected and used for impact assessment purpose and for feeding mobility planners and the processes they can follow to design new mobility policies.
One of the main objectives for MUV in respect to the SDGs is also to provide a sensitive impact structure that will validate the MUV assessment method. The impact assessment method is structured through a systemic measure of the SDG goals starting from the tracking data collected. MUV impact assessment is based on the following research questions:
  • to what extent can MUV change citizens’ mobility behaviors?
  • to what extent can MUV lead to a reduction of urban car-dependency traffic?
  • to what extent can MUV reduce the perceived gap between the use of a private car and the use of other more sustainable transport modes?
  • how could different interactions among local stakeholders (i.e., citizens, local businesses, local authorities) affect the behavioral change envisaged by MUV?
  • what is the added value of data collected by MUV? How can such data be used to provide insights to policymakers and mobility planners?
MUV assessment method is framed in this research paper on the systemic evaluation structure of MUV impacts, according to the diverse spheres of sustainability.

2. Materials and Methods

The impact assessment framework has been developed to validate and measure the added value of the MUV experimental gamification approach under the perspective to structure a method that will become both scalable and replicable in other contexts, thus ensuring an ease acquisition of the whole evaluation approach from other cities.

2.1. The Impact Assessment Framework

In accordance with CIVITAS SATELLITE approach [19], the MUV assessment framework covers not only impacts (impact evaluation) but also the process evaluation. The latter’s final aim is to understand the barriers to MUV implementation, the actions to overcome such barriers, and the drivers to leverage on. The process evaluation is, therefore, synergic to impact assessment and instrumental in answering the research questions set above.
MUV’s assessment approach, as based on several steps, develops a method suitable for impacts and process assessment in urban contexts, accounting for their peculiarities of places and spaces, but, at the same time, allowing comparability of the results in the six pilots (2017–2020). In order to obtain a transparent and correct understanding of the impact and the measure, it is necessary that the evaluation in each individual city/neighborhood follows the same guidelines of evaluation, especially:
  • the indicators for measuring the MUV impacts have been selected to be comparable in all the pilot cities. This selection does not prevent cities having their own additional local indicators important for the impact assessment (outside the scope of the project), but only the proposed set of impact indicators guarantees consistency in all the cities;
  • the methods of measurement of indicators in cities are aligned, allowing to reveal differences in results.
Special attention is paid to the identification of a set of indicators for impact evaluation (see Section 2.3 for details) according to well-grounded guiding principles. These principles are focused on (i) the transformation of MUV objectives from general indicators to comprehensive indicators, (ii) the coverage of the indicators are thought in relation to meaningful impacts derived from the MUV experimental gamification, (iii) the selection of indicators focus both on the availability of existing indicators and data, and deriving indicators from the MUV’s objectives themselves.
Since indicators are measured to orient progress toward goals, an overall guiding principle is that suitable indicators have been selected by capturing the essence of MUV objectives. This leaves aside the fact that some indicators might be available/already in use for a targeted phenomenon or not. As a matter of fact, as detailed in [20], it is essential to link the indicators to the objectives with future monitoring and evaluation activities; without clear objectives, it is not possible to monitor and evaluate whether an innovative action is on track.
With this in mind, the following indicators for impact evaluation have been selected according to the following MUV’s objectives:
  • OBJ 1: Sustainable urban mobility/new mobility culture: MUV promotes a shift towards more sustainable mobility in urban contexts; individual choices are at the core of impact evaluation on behavioral change approach to reduce urban vehicle traffic;
  • OBJ 2: Better health and environment: MUV raises citizens’ awareness on the quality of the urban environment and promotes healthier mobility choices, leading to a better environment;
  • OBJ 3: Evidence-based and human-centered urban mobility planning: MUV promotes the integration of people and personal mobility data into urban policy-making and planning processes at the neighborhood level;
  • OBJ 4: Foster local development: MUV is likely to generate positive spillover effects on the whole neighborhood and surroundings, even at the city level, involving local businesses and stimulating an innovative environment.
Moreover, the proposed assessment framework envisages the following four impact areas, that are well-grounded in the literature of impact evaluation in smart cities [19]:
  • IA–1 Society-People: it refers to the effects of the measure on the citizens living in the neighborhood and in the city, in terms of acceptability, mobility habits, perceived wellbeing, and new opportunities at the community level.
  • IA–2 Society-Governance: it refers to the effects of the measure on the way society is organized in terms of governance, e.g., planning and urban mobility policies.
  • IA–3 Economy: it focuses on the effectiveness and/or benefits derived from the measure in relation to the costs associated with its preparation, implementation, and operation, together with the economic spillover effects deriving from MUV implementation in the local development.
  • IA–4 Environment: it relates to the effects on the environment of reducing the use of private motorized transport, thanks to the measure, covering both polluting emissions and energy consumption.
Making use of the relationships between objectives and impact areas is the guiding principle to define the impact indicators that will guarantee that the resulting set of indicators will measure the effects meaningful to MUV and that they will cover different perspectives of the same result.
An overview table providing the list of impact indicators for each impact area and sub-area is shown in Table 2. The whole assessment framework is subject to a continuous review and refinement along the project lifetime in order to assess the feasibility of the baseline computation and of successive monitoring and evaluation; this flexible approach ensures a regular check and a continuous adjustment of the framework, catching new data opportunities, new trends at the societal level, and new policy and planning objectives of the six administrations.

2.2. Impact Evaluation

MUV impact evaluation is prospective, since it has been developed at the same time as MUV action has been designed, and baseline data have been collected prior to the implementation. Prospective evaluations have the best chance to generate valid counterfactuals since, at the design stage, alternative ways to estimate a valid counterfactual can be considered. The resulting impact evaluation is, thus, more likely to produce strong and credible evaluation results.
The impact evaluation follows a before-and-after comparison, as in Figure 2. The before-and-after comparison attempts to establish the impact of MUV by tracking changes in outcomes for program participants over time.
The ‘before’ situation is described by the baseline (see Section 2.4). Since the impact is the difference in outcomes for the same individual with and without participation in MUV, and since it is impossible to measure the same person in two different states at the same time (at any given moment in time, an individual either participated in MUV or did not), we encounter the so-called “counterfactual problem”: how do we measure what would have happened in the case of MUV absence? Although we can observe and measure the outcome for MUV participants, there is no data to establish what their outcomes would have been in the absence of MUV, that is the counterfactual.
Information about the counterfactual (i.e., what the outcome would have been in the six neighborhoods in case MUV action has not been yet fully implemented) is necessary in order to isolate the MUV impacts from the observed changes. Due to practical constraints, the counterfactual estimation in MUV is without a control group and envisages two different possibilities that will be alternatively selected depending on each impact indicator: A. constructing together with the involved stakeholders (e.g., pilot managers, mobility experts) a reference scenario, that, starting from the same baseline, could provide us with a likely counterfactual, or B. using the pre–MUV values at the baseline (t0) to estimate the post-MUV outcomes (counterfeit counterfactual). A slight adjustment of the second option has been introduced as a third alternative (option C) for the indicators related to the kms traveled on frequent routes (see Section 2.3 for more details about these indicators). In this latter case, the counterfactual estimate is provided by the travel behavior of all the pilot’s players on their most frequent routes as provided during their registrations to the MUV app (thus, not necessarily at t0). In this way, any behavioral change in their mobility patterns can be estimated by the difference between the observed behaviors and the counterfactual (i.e., the travel behavior they have declared when registering their frequent routes). Table 1 summarizes the possible options for MUV impact indicators’ counterfactual estimate; the choice made for the counterfactual estimate of each impact indicator is shown in the dedicated column of Appendix A Table A1.
During the monitoring phase, all the impact indicators defined in Table 2 are being collected and analyzed building on a defined monthly monitoring plan, allowing the evaluators to adjust and refine the impact assessment framework according to data and evidence collected.
Finally, in the ‘after’ evaluation, the impacts will be evaluated at the end of MUV action, by comparing outcomes with the counterfactual.

2.3. Impact and Context Indicators

The proposed assessment framework envisages two types of indicators: impact indicators (measuring the impacts generated by the action) and context indicators (providing information describing the geographical context of interest, i.e., the neighborhood/the city). Concerning the process evaluation, such assessment does not envisage process indicators for now, but it deals with a qualitative assessment of the whole process of implementation, investigating the enabling and inhibiting factors.
We are interested in impact indicators, rather than performance indicators, since—in order to answer to the above-mentioned research questions—the focus should be on to what extent a specific initiative, i.e., MUV, has had an impact on different aspects (e.g., society, economy, environment). Since there is not an ideal target of performance to be achieved, targets will not be defined for MUV impact indicators.
As far as the temporal classification of results is concerned, i.e., the results chain, the decision is to merge results in the unique category ‘impact’. The choice of aggregating the chains of causality is due to different reasons [21]: (i) after detailing the chain’s structure, it becomes evident that some chains (considered as minor) could be deleted; (ii) to be practical, the number of categories should not be excessive; (iii) a temporal classification of the results would have added greater complexity to the assessment framework. We do know that output and outcome indicators could be affected in the project lifetime, while impact indicators are likely to be affected after the project has been implemented and is in full use, which might take a few years. Nevertheless, (long term) impacts are included among the MUV indicators since they are a fundamental measure for reaching the project objectives, making it clear how progress toward strategic objectives will be assessed. Thus, from now on, we have referred to impact indicators as indicators of short, medium, and long term effects generated by the measure.
Various institutes and authorities have developed mobility indicators. Even though consensus on meeting the ‘triple bottom line’ exists (i.e., environmental, social, and economic sustainability), yet different indicator sets have been used to evaluate mobility measures in an urban context [22,23]. MUV impact indicators come from different initiatives, such as CIVITAS [19], CITYKeys [22], and TrafficO2 [24]; whenever necessary, tailor-made indicators have been designed. Regarding the data source of such indicators, Figure 3 summarizes the main data sources for MUV impact indicators: the MUV app, pilot managers, MUV monitoring stations, and local decision makers.
Table 2 presents the set of indicators, that is detailed in Table A1, including a special column aimed at specifying the data source of each indicator. The impact indicators have been classified in the four impact areas introduced in the impact assessment framework (see Section 2.1); moreover, some impact sub-areas have been identified to better organize the set of indicators.
Since the method aims to measure the impacts on the whole system (i.e., city/neighborhood), the impact indicators do not focus on a single individual. Obviously, the computation of some indicators need individual’s data (e.g., the indicators whose data source is the app), but their final value indicates an impact of the action on the neighborhood/city.
Many impact indicators whose data source is the MUV app relate to the kms traveled on frequent routes by the players of each pilot. The choice of computing such indicators only in relation to the frequent routes traveled relies on the fact that a real behavioral change will occur if the MUVers will change their daily mobility behaviors (i.e., the ones on their frequent routes), and not if they are only occasionally sustainable. Consider, for instance, an employee going to the workplace from Monday to Friday in his private car. Then, suppose on the weekend, he goes jogging and has an occasional bike ride, thus accumulating a lot of points on the MUV app. Not just because for this reason, his mobility habits could be considered ‘sustainable’. A real change in his mobility habits will rather be seen when he changes his mode of transport to go every day to the workplace, leaving his car at home and going to work, for example, by riding a bike.
Where possible, the choice has been to define ratio indicators, that are measurement units normalized to facilitate comparisons (e.g., per-year, per-capita, per-mile, per-trip, per-vehicle-year). Some indicators have been proxied by the app’s data, even though other indicators would have been more appropriate in case they were available. During the continuous update of indicators, some indicators can be added or modified whether new data sources become available.
Together with the impact indicators, also the set of context indicators has been developed. Context indicators provide information describing the geographical context of interest (the neighborhood/the city), and they are able to grasp the socio-demographic peculiar features of each neighborhood. They are introduced to facilitate the understanding of the neighborhood’s situation and, thus, of its impact evaluation. According to European Commission [25], context indicators usually deal with economic and financial fields (e.g., GDP, trade flows), social fields (e.g., demography, occupation, gender), and specific important sectors (e.g., education, health, environment). In the MUV case, special emphasis is put on transport and mobility.
The classification of MUV context indicators reflects this taxonomy, covering the following four areas (Table 3): socio-economic (C1–C19), transport (C20–C59), environment (C60–C61), and institutional (C62–C66). MUV context indicators have been collected in a collaborative way, directly involving the pilot coordinators both in the choice of indicators and in the collection of the data. Table A2 details the context indicators that have been collected in each MUV pilot. For each context indicator (row), its geographic level is provided (city/neighborhood/…) together with the corresponding data source. The involvement of the local stakeholders and of the pilot managers during this phase has been crucial, and we expect a continuous involvement of such actors during the evaluation so that the impact assessment results can help policy and decision makers in understanding the real impacts of the mobility measure.
Since, due to their nature, such indicators are not likely to drastically change during the project lifetime, an update of the values reported for each pilot in Table A3 will be performed—if necessary—during the project lifetime.

2.4. The Baseline

The MUV baseline defines the situation before MUV is implemented in each neighborhood. Operatively, the MUV baseline consists of a set of values for each impact indicator in each neighborhood before MUV comes into force. Appropriate baseline data is always critical for impact evaluation, as it is impossible to measure changes without reliable data on the situation before the innovative action begins.
One remark should be made for the baseline of the impact indicators computed from the app data related to the kms traveled for different transport mode by MUV players on average in each neighborhood (in IA1 ‘Society–people’ and IA–4 ‘Environment’). The baseline for such indicators is provided by data collected during the registration procedure of the players in the MUV app, during which the following information is required (compulsorily):
  • the modal split of each user on his/her most frequent route(s);
  • the length of such route(s);
  • the number of times per week he/she travels this route(s).
Furthermore, for the impact indicators in IA–4 ‘Environment’, further information is asked to the player to estimate his/her personal contribution to air emissions (i.e., CO2, CO, NOx, PMs):
  • in the case the player uses a car on the registered frequent routes, the kind of car she/he generally uses (segment: mini/small/medium/large; fuel: petrol/diesel/petrol hybrids/LPG/CNG/electric; EURO standard); then, COPERT model [26] is used to estimate the corresponding emissions in her/his urban context;
  • in the case the player uses a motorbike on the frequent routes, the kind of motorbike she/he usually drives (engine: 2 stroke/4 stroke; cubic capacity: <50 cm3/51–250 cm3/251–750 cm3/>750 cm3; EURO standard); then, COPERT model [26] is used to estimate the corresponding emissions in her/his urban context.
The baseline values herewith presented (Table A1) are based on the data collected in each pilot up to 31 December 2018, considering that the app has been officially launched three months before. The missing values (NAs) in the baseline are originated by the fact that the monitoring stations envisaged by the project have not been installed yet, and thus environmental data coming from such stations have not been collected, yet.

2.5. Process Evaluation

As introduced in Section 2.1, while the impact evaluation is aimed to quantify the added value of MUV action in the six neighborhoods along specific impact dimensions, the process evaluation—performed side by side with the impact evaluation—ensures a real understanding of the role the measure can have in a sustainable mobility strategy and provides insight into which elements are crucial to observed impact. MUV process evaluation is a qualitative analysis that deals with the evaluation of the activities of planning, implementation, and operation of MUV, in order to understand whether and why MUV has succeeded or failed in each of the six neighborhoods, with the final aim of understanding the barriers to MUV implementation, the strategies to overcome such barriers, and the drivers to leverage on. Together with the results of the impact evaluation, the outcome of the process evaluation will be the basis for the recommendations for other European cities interested in joining the sustainable mobility action.
The involvement of local stakeholders and pilot managers is crucial in the process evaluation, thanks to their thorough knowledge of the local context. Accurate process evaluation tables have been defined by the evaluators and are being collaboratively filled in by the pilot managers, in order to have a comprehensive understanding of the MUV process in each pilot considering each neighborhood’s peculiarities. A monitoring plan has been defined consisting of three measurements, once every about six months, after the app introduction.
Figure 4 shows an illustrative template table that pilot managers have to fill in during the ongoing process evaluation: for each step (activation of the community and management of the community), enabling and inhibiting factors are investigated per each stakeholder in terms of drivers, barriers, and lessons learned to be turned into action at the following iteration.
Section 3 provides some preliminary results of MUV process evaluation; a complete assessment will be performed at the end of the action, so as to provide recommendations useful for other cities and/or for other sustainable mobility actions.

3. Results

In the previous section, the general approach to evaluate a sustainable mobility measure has been discussed, using the case of MUV action as an example of application. In this section, the preliminary results are discussed, in terms of a set of impact indicators, criteria for their measurement, baseline, and counterfactual estimate. Since the monitoring of the action is still ongoing, the (ex-post) impact evaluation results cannot be presented yet, and they will be the object of future works by the authors.
Regarding the context indicators, many indicators have been requested at a neighborhood level, but—due to the unavailability of data at that geographical level—in some cases, they have been collected at a higher level (e.g., city). Not all requested data was available in each pilot; thus, some missing values are present (labeled as ‘unknown’). Some remarks could be made, observing some relevant context indicators in the different pilots (Table A3):
  • the six neighborhoods have a similar age structure of residents (C7, C8), with Ghent and Palermo having a slightly higher percentage of the young population than the other neighborhoods;
  • the average available income, measured as the disposable per-capita income of private households, is expressed in PPS (Purchasing Power Standard) to make the figures comparable between regions (Figure 5). The differences in such indicator (C9) are quite relevant, varying from 11,600 PPS in Fundao to 18,900 PPS in Ghent;
  • the social structure is different also in employment terms (C16), going from 3.4% of unemployment rate in Ghent to 21.5% of unemployment rate in Palermo (Figure 5);
  • the rate of smartphone ownership (C17) will be an important context indicator to take into account during evaluation activities, since using a smartphone constitutes a prerequisite to installing the MUV app. Apart from Fundao (67%), this rate is greater than 70% in all the pilots, with a peak equal to 87% in Amsterdam and Barcelona (Figure 6);
  • even though the smartphone ownership rate is quite aligned with the other pilots, Palermo constitutes an exception for the use of the internet on the move (C18, C19). Only 72% of individuals aged 16–24 use mobile devices via mobile or wireless connection in Palermo, against over 92% of the other sites (Figure 6). The difference is even more evident getting older, with 11% of individuals aged 55–74 using the internet on the move, against over 50% of the other pilots (with the exception of Fundao);
  • the average waiting time at a bus station (C25) can heavily influence the city travelers to move with public transport. The six pilots show clear differences in waiting times, with users of public transport in Helsinki waiting an average of 4 min at a standstill, against a 23–minute wait for a Palermitan;
  • the traffic congestion in the cities (C46–C54) is quite different among the pilots. Consider, for example, the congestion level during evening peaks (C49), i.e., the increase in evening peak travel times when compared to a free flow situation. Palermo reaches 69% of congestion level in the evening peak (i.e., during the evening peak, it takes 69% of the extra time to travel the same route compared to a free flow situation) against the 34% of Ghent;
  • another important factor to take into consideration when choosing the travel mode is the rate of car ownership (C36), which is among the highest in Europe in Palermo (625 cars owned per 1000 inhabitants), while it is below the European average in Amsterdam (481 cars per 1000 inhabitants). Fundao and Palermo are, among the pilots, the sites in which the gasoline is more expensive (C44) in relation to the average available income (Figure 7);
  • considering the road safety dimension (C55–C59), the numbers show a huge difference in people killed in road accidents in the six sites (Figure 8). Helsinki is the safest city among the pilots (0.048 people killed per 10,000 population), while Fundao is the least secure one (1.07 people killed per 10,000 population). Unfortunately, for many pilots, it was not possible to collect information from the pilot managers about the quantity and type of road accidents (e.g., car-car/car-bike/car-people);
  • as shown in Figure 9, the six cities are very different for their weather conditions (C60, C61), that could constitute a relevant factor in the mobility choice of each individual (e.g., walk, bike). Palermo is the hottest pilot and among the least rainy cities (average annual temperature: 18 °C; the average number of days with precipitation in the year: 74.3). In Ghent, it rains two days out of three (221 days of precipitation in one year), while Helsinki is the coldest city (average annual temperature: 5 °C).
Regarding impact indicators, Table A1 provides an overview of MUV impact indicators, together with their updated baseline values and the type of counterfactual estimate established for each indicator (for the counterfactual legend, please refer to Table 1). More detailed indicator definition sheets have been developed to serve as practical information and use guidelines for each impact indicator.
A Python code has been written for the computation of impact indicators, in order to be integrated within the architecture framework. The implemented Python code is composed of three modules. The first module returns, for each most frequent route registered by a player, a) the latitude and the longitude of the origin and of the destination; b) the weekly distance traveled [km] and travel time [minutes] in each transport mode (walk, bike, public transport, carpooling, car, motorbike) on that route, that will be subsequently used as counterfactual (reference scenario), in accordance with the frequency and the modal split declared by the player when registering the frequent route. The second module computes the weekly value of the basic indicators that are related to kms traveled each week on frequent routes (e.g., km traveled by car in each pilot). This code implements an algorithm that estimates the kms traveled in each transport mode by each player on the registered frequent routes. As a matter of fact, only sustainable transport modes can be registered through the MUV app (i.e., walk, bike, public transport), and it is, thus, necessary to estimate the kms traveled by each player in private car and/or motorbike on these routes. Furthermore, this algorithm provides some adjustments to make up for forgotten tracked routes. Such estimates are based on the information provided by the user while registering each most frequent route. The last module is the code running every month (according to a defined monitoring plan) that returns the (monthly) values of impact indicators. In this code, also the counterfactual estimate is performed for the indicators related to the kms traveled on frequent routes in the pilots.
Regarding the baseline presented in Table A1, the authors are absolutely aware that the users of the MUV app are not a representative sample of the target population, but some important considerations emerge. An indicative example could be Palermo’s indicator IA1.S2.3 “Modal split”, defined as the percentage of kms traveled using each transport mode (private car, walk, bike, public transport, motorbike, carpooling) on frequent routes by Palermo’s players. Figure 10 shows the great difference between the modal split of Palermo provided by the city administration (representative of the city) and information collected by Palermo players’ mobility habits during the registration phase to MUV app (baseline).
As shown in Figure 10, people providing the baseline for MUV have very different mobility habits with respect to Palermo population, at least on frequent routes. As a matter of fact, Palermo’s MUV players exhibit high sustainable mobility habits (IA1.S2.1 “Sustainable mobility habits” = 53.97%), and the use of not-active transport is very low (car = 32.22%; moto = 11.05%; carpooling = 2.76%) compared to Palermo in general (private transport = 78%). This factor confirms that, based on the data currently available, MUV players are likely to be more sensitive to the sustainable mobility issues, and therefore constitute a sample slightly deviated from the average population of the city.
As far as the process evaluation is concerned, the first preliminary shortcomings relate to the step of activation of the neighborhood community, since the management of the community itself has not been investigated yet due to the timings of the monitoring plan. In this regard, Figure 11 summarizes the most prominent results of the first monitoring in the pilots.
As depicted in Figure 11, the success factors (i.e., the drivers) that cross almost all the pilots cover the social and political/institutional sphere, showing, on the one hand, the great relevance of effective communication activities to make people aware and involved in the sustainable mobility initiative, and, on the other, the importance of a massive involvement of the municipality to successfully engage the community. On the contrary, currently the barriers to the implementation of the initiative are mainly technological: the users declare that the app enrolment is too long and too complicated, too many bugs have been detected, and the ‘yet–another–app’ syndrome is difficult to overcome. These factors have led many users to a feeling of frustration, which potentially takes them away from the community. Thanks also to the surveys carried out within the process evaluation, during the first months after the launch of the MUV app has undergone numerous releases to make some improvements to the app enrolment and to fix some detected bugs, precisely to meet some questions raised by the users. Furthermore, each pilot has defined some priority takeaways to be turned into action in the following iteration, in accordance with the strong points and the peculiarities of each local context.
The monitoring of the MUV process and impact indicators is ongoing, and future works will deal with the results of ex-post impact and process evaluation within the framework described here.

4. Discussion

MUV evaluation approach, hereby described, effectively addresses and potentially provides answers to the questions raised at the end of Section 1 for diverse practical reasons.
The first is that the overarching framework for evaluation accounts both for impact and process evaluation, thus, leading to a both qualitative and quantitative assessment that attempts to capture the quantification of the outcomes (e.g., reduction of urban vehicle traffic), as well as the understandings related to the implementation of the solution (e.g., how supporting activities could affect MUV outcomes). Moreover, the relationships between MUV objectives and impact areas (Society-People, Society-Governance, Economy, and Environment) are used as a guiding principle to define the impact indicators. This results in a set of indicators that are meaningful for urban contexts to measure experimental gamification issues and suitable measures on the effects that are selected to be relevant. For example, in MUV, these outcomes generate data that will potentially cover different perspectives of the same result. Then, continuous involvement of the local communities during the evaluation process opens a possibility for co-monitoring with local stakeholders on the impact assessment to respond to their expectations. Thus, MUV impact evaluation results can lead to concrete and effective insights for urban policy and decision makers and urban mobility planners. Furthermore, the users’ mobility data and environmental conditions collected through MUV initiative enable more informed policy decisions for urban futures.
The MUV context indicators show how the six pilots are inherently different, and, thus, indicate a likely difference in response to MUV initiative. Such considerations about context indicators will be qualitatively used by impact evaluators to interpret and motivate outcomes that will be measured in each pilot. In this way, it would be possible to include also ongoing mobility initiatives (and/or infrastructure changes) within the neighborhood and the city (context indicators C65, C66). (context indicators C65, C66). The combination of the baseline and the context indicators provides an overall picture of the situation in each pilot before the MUV initiatives come into action. Such a picture can frame the data ‘before’ the implementation of MUV impact evaluation; at the end of monitoring, MUV outcomes can be quantified and, thus, impacts will be evaluated.
The set of 40 MUV impact indicators (Table 2) has been pursued also to be utilized during the MUV co-creation process to support both the pilot managers and the policy-makers in each pilot city towards a ‘conversational planning’ with the citizens.
As detailed in the previous sections, the evaluation approach herewith described is scalable and replicable, to allow consistency and comparability among pilot cities. Several other European cities have been already chosen to join MUV initiative. The method illustrated here is meant to be applied to other urban contexts to reflect on the real effects of these experimental gamification experiences with an aim to feed a sustainable movement for the future.

Author Contributions

Conceptualization, B.C. and A.V.; Data curation, B.C.; Formal analysis, B.C.; Funding acquisition, S.D.D.; Investigation, B.C.; Methodology, B.C.; Project administration, A.V. and S.D.D.; Software, B.C.; Supervision, A.V. and S.D.D.; Validation, B.C.; Visualization, B.C.; Writing–original draft, B.C., N.M., and E.L.; Writing–review & editing, B.C., N.M., E.L., A.V., S.D.D., and S.M.


This research was funded by the European Union’s Horizon 2020 research and innovation program, under grant agreement No 723521.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Appendix A

This appendix contains details about MUV impact indicators (Table A1) and MUV context indicators (Table A2) in the six neighborhoods.
The impact indicators codes of Table A1 refer to Table 2, while the counterfactual estimate column refers to the legend provided in Table 1. The context indicators codes of Table A2 refer to Table 3.
Table A1. Mobility Urban Values (MUV) impact indicators: description, baseline, type of counterfactual estimate, and source (AMS = Amsterdam; BCN = Barcelona; FUN = Fundao; GEN = Ghent; HEL = Helsinki; PAL = Palermo).
Table A1. Mobility Urban Values (MUV) impact indicators: description, baseline, type of counterfactual estimate, and source (AMS = Amsterdam; BCN = Barcelona; FUN = Fundao; GEN = Ghent; HEL = Helsinki; PAL = Palermo).
Indicator (Code, Name, Unit of Measurement, Short Description)Baseline
Data Source
IA1.S1.1 Awareness level [%]
% of people in the pilot with knowledge of MUV
AMS 0.13%
BCN 19.31%
FUN 0.00%
GEN 10.14%
HEL 36.43%
PAL 38.44%
Bpilot managers
IA1.S1.2 Involvement level [%]
% of people in the pilot involved in the co-creation activities and/or other MUV-related activities
AMS 50.37%
BCN 4.47%
FUN 0.00%
GEN 15.17%
HEL 0.41%
PAL 3.60%
Bpilot managers
IA1.S1.3 Acceptance level [%]
% of people in the pilot registered to MUV app
AMS 11.03%
BCN 0.21%
FUN 0.00%
GEN 0.67%
HEL 0.05%
PAL 0.37%
BMUV app
IA1.S1.4 Activeness level [%]
% of active players in the pilot
0 (all the neighborhoods)BMUV app
IA1.S1.5 Perseverance level [%]
% of players continuously active in the year
0 (all the neighborhoods)BMUV app
IA1.S2.1 Sustainable mobility habits [%]
% of kms traveled in a sustainable way (walk/bike/public transport) on frequent routes
AMS 84.77%
BCN 81.78%
FUN 23.66%
GEN 73.27%
HEL 65.95%
PAL 53.97%
CMUV app
IA1.S2.2 Use of private car [%]
% of kms traveled by private car on frequent routes
AMS 6.21%
BCN 10.12%
FUN 47.03%
GEN 20.37%
HEL 29.33%
PAL 32.22%
CMUV app
IA1.S2.3 Modal split [%]
% of kms traveled using each mode (walk, bike, public transport, private car, motorbike, carpooling) on frequent routes (6–elements array)
AMS 16.90%; 43.24%; 24.63%; 6.21%; 4.51%; 4.51%
BCN 24.04%; 11.20%; 46.53%; 10.12%; 7.17%; 0.94%
FUN 17.50%; 2.39%; 3.77%; 47.03%; 12.32%; 16.99%
GEN 9.28%; 49.39%; 14.60%; 20.37%; 4.54%; 1.81%
HEL 15.30%; 17.27%; 33.38%; 29.33%; 2.36%; 2.36%
PAL 27.42%; 11.23%; 15.32%; 32.22%; 11.05%; 2.76%
CMUV app
IA1.S2.4 Travel time [minutes/day]
average daily time spent traveling using each mode (walk, bike, public transport, private car, motorbike, carpooling) on frequent routes (6–elements array)
AMS 64.7; 33.1; 23.2; 4.2; 5.4; 5.4
BCN 90.6; 18.5; 41.6; 11.1; 15.5; 3.9
FUN 59.4; 20.1; 21.9; 15.3; 7.9; 9.4
GEN 50.9; 32.4; 20.9; 9.3; 4.8; 3.3
HEL 32.3; 13.1; 20.0; 16.1; 5.1; 5.1
PAL 39.8; 26.4; 20.9; 13.7; 13.6; 5.3
CMUV app
IA1.S3.1 Community cohesion among travelers [people/car]
level of contact between people living in the community, perception of being part of their community. Proxy: average carpooling vehicle occupancy
NA (the app does not have the carpooling functionality, yet)-MUV app
IA1.S4.1 Physical activity [cal/person*week]
physical activity performed via active transport (walk and bike) on frequent routes. Proxy: average calories burned on frequent routes
AMS 1971.21
BCN 1810.68
FUN 1326.70
GEN 1518.33
HEL 877.33
PAL 849.63
CMUV app
IA2.S1.1 Planning process [5–level Likert scale]
changes in the process to develop mobility plans, thanks to MUV, in terms of strategic level vision, level of public involvement, sector integration, institutional cooperation, monitoring and evaluation, finance, implementation. Linked to the indicator “public investments” (IA3.S2.1)
1 (all the neighborhoods)Alocal decision makers
IA2.S1.2 Quality of policies, plans, and programs [5–level Likert scale]
qualitative evaluation of the change in the process to develop policies, plans, and programs
1 (all the neighborhoods)Alocal decision makers
IA2.S2.1 Rules and regulations [5–level Likert scale]
the extent to which MUV has contributed to, or inspired, changes in rules and regulations (i.e., if MUV is able to change the context in which it is applied, by providing a different interpretation of existing rules and regulations)
1 (all the neighborhoods)Alocal decision makers
IA2.S2.2 Policies [5–level Likert scale]
the extent to which MUV has contributed to, or inspired, changes in the current urban mobility policies (e.g., update SUMP (sustainable urban mobility plan))
1 (all the neighborhoods)Alocal decision makers
IA2.S2.3 Policy making process [5–level Likert scale]
the extent to which MUV has contributed to, or inspired, changes in the process to develop policies and programs, in terms of strategic level vision, level of public involvement, sector integration, institutional cooperation, monitoring and evaluation, finance, implementation
1 (all the neighborhoods)Alocal decision makers
IA2.S2.4 Finance [5–level Likert scale]
the extent to which MUV has contributed to—or inspired—the development of new forms of financing of mobility solutions
1 (all the neighborhoods)Alocal decision makers
IA2.S2.5 Cooperation structures with stakeholders [5–level Likert scale]
the extent to which MUV has changed the cooperation structures between public and private stakeholders to develop and implement sustainable mobility solutions
1 (all the neighborhoods)Alocal decision makers
IA2.S3.1 Quality of open data [5–level Likert scale]
the extent to which MUV has changed the level of quality of mobility open data
1 (all the neighborhoods)Alocal decision makers
IA2.S3.2 Open datasets [#]
# of open mobility (government) datasets born, thanks to MUV
0 (all the neighborhoods)Alocal decision makers
IA3.S1.1 Global sponsors involvement [%]
involvement of global sponsors in the community (global sponsor: organization providing goods and services globally or nationally)
0 (all the neighborhoods)Bpilot managers
IA3.S1.2 Community interaction with global sponsors [%]
level of interaction of the community with the global sponsors. Proxy: # check–in (and all CTA–call–to–action, if any) at global sponsors/# active players
0 (all the neighborhoods)BMUV app
IA3.S1.3 Local sponsors involvement [%]
involvement of local sponsors in the community (local sponsor: public or private organization providing goods and services inside the neighborhood)
0 (all the neighborhoods)Bpilot managers
IA3.S1.4 Community interaction with local sponsors [%]
level of interaction of the community with the local sponsors. Proxy: # check–in (and all CTA–call–to–action, if any) at local sponsors/# active players
0 (all the neighborhoods)BMUV app
IA3.S2.1 Public investments [€]
amount of investments of the municipality on new mobility initiatives, thanks to MUV
0 (all the neighborhoods)Alocal decision makers
IA3.S2.2 Private investments [€]
amount of investments of the sponsors on MUV and MUV-related initiatives
0 (all the neighborhoods)Aglobal/local sponsors
IA3.S3.1 Innovative environment [5–level Likert scale]
the extent to which MUV increases the level of innovativeness of the urban environment, in terms of exploiting new mobility-related opportunities for helping enterprises to innovate or innovate more
1 (all the neighborhoods)Alocal decision makers
IA3.S3.2 Economic activity [5–level Likert scale]
the extent to which MUV impacts the economic activity of the pilot, in terms of, for instance, job creation and additional economic activity (e.g., creation of leisure-based networks)
1 (all the neighborhoods)Alocal decision makers
IA3.S3.3 Open data exploitation [#]
third-party developments: number of apps/services/API calls developed by third parties from MUV open data
0 (all the neighborhoods)Apilot managers
IA4.S1.1 CO2 emissions from road traffic [g/km]
average emissions of CO2 per km traveled on frequent routes
AMS 1971.2
BCN 1810.7
FUN 1326.7
GEN 1518.3
HEL 877.3
PAL 849.6
CMUV app
IA4.S1.2 CO2 level [ppm]
the concentration of CO2 in the neighborhood (still unsure that monitoring stations will measure this value)
NA-MUV monitoring stations (still unsure)
IA4.S2.1 Noise level [dB]
level of noise in the neighborhood
NA-MUV monitoring stations
IA4.S2.2 NOx emissions from road traffic [g/km]
average emission factors of NOx per km traveled on frequent routes
AMS 0.104
BCN 0.201
FUN 0.189
GEN 0.102
HEL 0.182
PAL 0.161
CMUV app
IA4.S2.3 NO2 level [µg/m3]
the average concentration of NO2 in the neighborhood
NA-MUV monitoring stations
IA4.S2.4 PM2.5 emissions from road traffic [mg/km]
average emissions of PM2.5 per km traveled on frequent routes
AMS 0.752
BCN 2.300
FUN 1.958
GEN 1.512
HEL 4.079
PAL 6.490
CMUV app
IA4.S2.5 PM2.5 concentration [µg/m3]
the average concentration of PM2.5 in the neighborhood
NA-MUV monitoring stations
IA4.S2.6 PM10 concentration [µg/m3]
the average concentration of PM10 in the neighborhood
NA-MUV monitoring stations
IA4.S2.7 CO emissions from road traffic [g/km]
average emissions of CO per km traveled on frequent routes
AMS 0.500
BCN 0.462
FUN 0.764
GEN 0.748
HEL 0.363
PAL 0.893
CMUV app
IA4.S2.8 CO level [ppm]
the average concentration of CO in the neighborhood (still unsure that monitoring stations will measure this value)
NA-MUV monitoring stations (still unsure)
IA4.S3.1 Energy consumption from road traffic [kgoe/km]
average energy consumption per km traveled on frequent routes
AMS 0.013
BCN 0.019
FUN 0.045
GEN 0.019
HEL 0.032
PAL 0.029
CMUV app
Table A2. Context indicators: geographic level and their data source.
Table A2. Context indicators: geographic level and their data source.
CodeContext IndicatorGeographic LevelData Source
C1NUTS3 code--
C2Neighborhood nameneighborhoodpilot coordinator
C3Area dimensionneighborhoodpilot coordinator
C4Population of the neighborhoodneighborhoodpilot coordinator
C5Population of the citycityUrbiStat
C6Population density in the neighborhoodneighborhoodcomputation
C7Age structureneighborhoodpilot coordinator
C8Age structurecitypilot coordinator
C9Average available income (yearly)NUTS2EUROSTAT (2014)
C10Driving age populationcityUrbiStat
C11Business densityneighborhoodpilot coordinator
C12Landmarks and historic featuresneighborhoodpilot coordinator
C13Shopping areas/commercial centersneighborhoodpilot coordinator
C14Neighborhood assetsneighborhoodpilot coordinator
C15Employment rateNUTS2EUROSTAT (2017)
C16Unemployment rateNUTS2EUROSTAT (2017)
C17Smartphone ownershipcountryGoogle Consumer Barometer
C18Internet use on the move (total)countryStudy (2017)
C19Internet use on the move (by age group)countryEUROSTAT (2017)
C20Public transportneighborhoodEUROSTAT (2017)
C21Public transportneighborhoodpilot coordinator
C22Public transportneighborhoodpilot coordinator
C23Public transportneighborhoodpilot coordinator
C24Public transportneighborhoodpilot coordinator
C25Public transportcitypilot coordinator (Moovit where available)
C26Public transportcitypilot coordinator
C27Public transportcitypilot coordinator
C28Public transportcitypilot coordinator
C29Public transportneighborhoodpilot coordinator
C30Bikeneighborhoodinfo will be derived from the maps
C31Bikeneighborhoodinfo will be derived from the maps
C33Bikecitypilot coordinator
C34Walkneighborhoodpilot coordinator
C35Walkneighborhoodinfo will be derived from the maps
C36Private carcountryComputation
C37Private carcity, otherwise countryEUROSTAT (2016)
C38Shared servicescitypilot coordinator
C39Shared servicescitypilot coordinator
C40Shared servicescitypilot coordinator
C41Shared servicescitypilot coordinator
C42Shared servicescitypilot coordinator
C43Transport costscitypilot coordinator
C44Transport costscountrypilot coordinator
C45Transport (11 June 2018)
C46Traffic congestionneighborhoodpilot coordinator
C47Traffic congestioncitypilot coordinator
C48Traffic congestioncityTomTom traffic index where available, otherwise pilot coordinator (Fundao)
C49Traffic congestioncityTomTom traffic index where available, otherwise pilot coordinator (Fundao)
C50Traffic congestionneighborhoodTomTom traffic index where available, otherwise pilot coordinator (Fundao)
C51Traffic congestionneighborhoodpilot coordinator
C52Traffic congestioncitypilot coordinator
C53Traffic congestioncitypilot coordinator
C54Traffic congestioncitypilot coordinator
C55Safety and securitycitypilot coordinator
C56Safety and securitycityEUROSTAT (2016)
C57Safety and securitycitypilot coordinator
C58Safety and securitycitypilot coordinator
C59Safety and securitycitypilot coordinator
C60Weather conditionscitypilot coordinator
C61Weather conditionscityWeatherBase
C62Sustainable urban mobility plan (SUMP)cityWeatherBase
C63Easementscity/neighborhoodpilot coordinator
C64Mobility policiescitypilot coordinator
C65Mobility initiatives/infrastructure already plannedneighborhoodpilot coordinator
C66Mobility initiatives/infrastructure already plannedcitypilot coordinator
Table A3. Values of context indicators in the six neighborhoods (at October 2018).
Table A3. Values of context indicators in the six neighborhoods (at October 2018).
C1NL329 (NL326 in NUTS2013)ES511PT16HBE234FI1B1ITG12
C2BuitenveldertSant Andreu de PalomarOld TownMuide–MeulestedeJätkäsaariCentro storico (Old town)
C70–14: 12.3%
15–64: 66.0%
65–74: 11.3%
75+: 10.4%
0–14: 13.2%
15–64: 66.6%
65–74: 10.2%
75+: 10.0%
0–14: 14.91%
15–64: 66.23%
65–74: 9.85%
75+: 9.01%
0–14: 21,9%
15–64: 67,6%
65–74: 5,3%
75+: 5,2%
unknown0–14: 17.6%
15–64: 70.0%
65–74: 6.3%
75+: 6.1%
C80–14: 15.6%
15–64: 72.6%
65–74: 6.9%
75+: 4.8%
0–14: 12.6%
15–64: 65.8%
65–74: 10.1%
75+: 11.5%
0–14: 11.75%
15–64: 60.55%
65–74: 13.09%
75+: 14.61%
0–14: 16.3%
15–64: 67.1%
65–74: 7.9%
75+: 8.7%
0–14: 14.0%
15–64: 69.4%
65–74: 9.7%
75+: 6.9%
0–14: 14.4%
15–64: 66.5%
65–74: 10.2%
75+: 8.9%
C11177 registered firms (13.27 firms per 1000 active people)in the CITY: 178,607 registered firms889unknownunknownunknown
C12Hortus Botanicus VU- Historical memory (6 buildings)
- Industrial and architectural heritage: creation factories, cultural centers
- Pd: app Rutes Sant Andreu
- Municipal Museum
- Moagem
- old harbor site: old crane, 19th-century harbor hangars, two small churchesNone- 95 churches
- 40 convents
- more than 400 aristocrats palaces
C13- shopping mall ‘Gelderlandplein’
- shopping center Rooswijk
- shops in Zuidas
- commercial streets
- markets
municipal market (Saturday)
open-air market (Monday)
commercial streets in Old Town
- less than 10 cafes and restaurants
- grocery stores (mainly run by Turkish people)
- municipal market (Saturday)
- (home appliances and electronics outlet store)- 4 historic markets
- the high concentration of commercial activities (3 main commercial streets)
C14- 46,700 people working in the neighborhood
- business district Zuidas (78% of the people working in the neighborhood)
- 3 city parks
- sport clubs
- low key meeting facilities
active networks international schools
Strong associative community (262 associations registered).
Community centres Numerous sport clubs Recreation Education and schools
Churches/Religious associations
Active networks
- Fablab (social/business incubator and co–working space)
- Living Lab square: concerts and cultural events
- two parks
- Library
- Bars
- Moagem, cultural center
- Octogono, concert hall
2 schools, neighborhood centre, community centre, small youth centre, cultural centre (usually for small concerts/theatre plays), church might also get new possibilities for cultural events, small library mainly for kids and elderly people in local community centre- passenger port
- Kaapelitehdas event arena
- Hotel Clarion (conference center, 16th floor sky bar)
- 7 theaters
- nightlife (restaurants, pubs, bars)
C1916–24: 98%
25–54: 92%
55–74: 72%
16–24: 95%
25–54: 89%
55–74: 49%
16–24: 95%
25–54: 69%
55–74: 24%
16–24: 92%
25–54: 84%
55–74: 52%
16–24: 96%
25–54: 91%
55–74: 55%
16–24: 72%
25–54: 35%
55–74: 11%
C25unknown10 minNA (timetabled buses).15 min4 min23 min
C26220.384985,066 millions trips in year 2017 (total)/365 days = 2699 millions daily tripsunknownunknownunknown280,000
C27204.98 users/km used
166.98 users–km/1000
Average journey in Km: 2.80
(191.22 Km bus lanes in the city)
(3388.12M bus places/km)
C28100% diesel buses
(2% euro IV, 72% euro V, 26% euro VI)
45.71% diesel buses (lower than Euro III)
34.84% of CNG buses
0.37% of electric buses
19.08% hybrid buses (diesel-electric/CNG/electric)
100% diesel buses
57% diesel buses (legisl. std)
43% other (hybrid buses)
Remark that there are also many trams in Ghent (tram: 28% of the total amount of driven km)
C29info will be derived from the mapsinfo will be derived from the mapsinfo will be derived from the mapsinfo will be derived from the mapsinfo will be derived from the mapsinfo will be derived from the maps
C30info will be derived from the mapsinfo will be derived from the mapsinfo will be derived from the maps9 km cycling infrastructure2 km of cycle lanes4.5 km of cycle lanes
C32unknownin the CITY: 27,281 (26,158 park places on the street/1123 underground parking)25 places41 placesunknownunknown
C33780 (from age 12+)106,453 biking users/6000 bikes (public)
165,499 daily trips (43,836 daily trips by biking)
unknown823 have at least one bike (stadsmonitor)unknownunknown
C34info will be derived from the mapsinfo will be derived from the maps- 220 m of pedestrian streets
- no pedestrian squares
info will be derived from the maps- 1 km of pedestrian streets
- no pedestrian squares
- 175,487 m2 of pedestrian squares
C37unknown1.61.2 (country)1.12
(based on MobOnderzoek 2015 - aantal autobestuurders en auto inzittenden)
C384000 taxis10,522252201500unknown
C39Yes (Uber)Yes (Cabify)NoNoNoNo
C40Yes (Toogether)NoNot structuredYesNoNo
C414233 cars399 VTC (chauffeur–driven vehicles) licenses/driving age peopleNo2.04250 vehicles- 85 car sharing parking lots (300 parking spaces)
- 140 cars
C42- 2 bike sharing stations, 7 fast/easy bike rentals at train stations (not for tourists)
- an unknown number of bikes
- 465 bike sharing stations (45 are electric ones)
- 6300 bikes (300 are electric ones)
(excluding bike sharing system of national railway company) (Trapido)
150 stations, 2550 bicycles -> 0.004 bicycles per inhabitant- 35 bike sharing stations
- 238 bikes
C43€ 47.00€ 54.00unknown€ 53.00€ 54.70€ 32.00
C44€ 1.68€ 1.33€ 1.60€ 1.48€ 1.56€ 1.62
C45€ 1.40€ 2.25€ 0.90€ 0.80€ 4.00€ 1.00
C4617–188–9; 17–20no (just parking)8–9; 16:30–18several short ones related to ferry schedules8–9/17–19
C5013433in the CITY: 114,105 for cars + 68,250 for motorbikes + 10,513 good distribution vehiclesunknown2301unknownunknown
C51764 (public parking space)in the CITY: 591,612 off-street parking spacesunknown0unknown1103
C5634741698 (2017)80 (2017)unknownunknown2256 (2016) The Palermo municipality provide only the aggregated data related to the number of accidents occurred.
C57188601 (2017)unknownunknownunknown
C5896650 (2017)6 (2017)unknownunknown
C59621411 (2017)unknownunknownunknown
C6110.0 °C15.5 °C15.7 °C11.0 °C5.0 °C18.5 °C
C62YesYesYesYesYesUnder development
C63- subsidies for electric cars
- the network of charge–stations in the public (and non–public) space
- environmental zones for trucks, work-buses, taxis, and scooters
- a lot of areas with max. 1 parking permit per household
- park and ride facilities
- Subsidies, free surface public on the street and free tolls to buy an electric vehicle
- Subsidies for electric bikes purchase (PIVE plan–AMB)
- Awareness campaigns (motorcycles, Workplace Road Safety Campaign)
- Incentives to renovate or sell old cars (T–Verda)
- neigh. inhabitants have a free parking pass (for one car)
- national incentives for electric cars
- neigh. inhabitants have a free parking pass (for one car)
- subsidies for electric car sharing, bike sharing, electric bikes, CNG
50% discount on parking for low–emission vehicles- Carsharing vehicles have access to the restricted traffic area and can travel along taxi lanes.
- The first half hour of use of the bike sharing system is always free.
C64- policy priorities for bikes (direct and smooth bike routes, removal of unused bikes in the parking, new biking policy measure)
- special attention to road safety (in particular for vulnerable users)
- stimulate usage of bikes
- Reduce the number of accidents
- Facilitate modal shift towards more sustainable modes.
- Reduce air and noise pollution resulting from transportation
- Reduce energy consumption in transportation
- Ensure accessibility to the mobility system.
- Increase the efficiency of transportation systems.
- parking meters to control parking spaces → little impact due to the lack of enforcement (little control of who pays and who doesn’t).- increasing the car–parking zone outside the city center- 5-level traffic planning guideline (pedestrians and cycling are a top priority, followed by public transport, logistics and only as of the last priority the private car traffic)- Limited Traffic Zone (ZTL) in Centro Storico
- Two free shuttles that connect the key areas of the old town with the central railway station
- Pedestrianization process of streets and squares
- Enhancement of bike and car sharing stations
C65Not at a neighborhood level.Improve the infrastructure for bus and bikes:
increase of bike lanes km, increase of bus lines and of bus lanes km

Remarkable works in street “Carrer Gran” (May 2018–May 2019)
construction of cycling paths (in 2018)
construction of new bike parking spaces
- new bridge replacing the old one (2022)
- new bridge in the south (construction starts in 2019)
- the whole neighborhood will become paid parking (not clear yet when)
- new cycling and pedestrian areas along the canal
- new housings planned
- the former logistics port is being converted into a residential area, traffic planning & construction related to ferry terminal ongoing till 2020
- two more indoor parking facilities (by 2020)
- multiple mobility service trials starting summer 2018
- Railway ring under construction with two stops inside the old town
- New bicycle lanes planned
C66More railways will be built and therefore more trains will run. Several streets will become biking streets with the car being guest. In July ‘18 a new underground line is being opened, making a fast connection from north to south.- Organization of the city’s urban pattern in superblocks
- Implementation of the new orthogonal bus network
- Development of the cycling network
- Promotion and positive discrimination measures of high occupancy vehicles
- Review of the regulation of parking on and off road
-- car sharing
- cycling infrastructure
- Low Emission Zone (from 01/01/2020)
- a western extension to metro (end of 2017)
- bicycle highways (year?)
- new tram lines (year?)
- new legislation from June 2018 (major deregulation) will pave way for new transport services, including sharing services (e.g., Lyft, Uber)
- Restricted Traffic Zone (ZTL) also outside Centro Storico
- Carsharing fleet (IoGuido) will reach 555 vehicles by 2020
- Extension of cycling paths planned by PGTU, from the current 20 km to 145 km


  1. United Nations General Assembly. Transforming Our World: The 2030 Agenda for Sustainable Development (A/RES/70/1), 21 October 2015. Available online: (accessed on 9 April 2019).
  2. Wendel, S. Designing for Behavior Change: Applying Psychology and Behavioral Economics, 1st ed.; O’Reilly Media, Inc.: Sebastopol, CA, USA, 2013. [Google Scholar]
  3. Strengers, Y.; Maller, C. Social Practices, Intervention and Sustainability—Beyond Behaviour Change, 1st ed.; Routledge: Oxon, UK, 2015. [Google Scholar]
  4. Caillois, R. Man, Play and Games; University of Illinois: Urbana, IL, USA, 2001. [Google Scholar]
  5. Deterding, S.; Sicart, M.; Nacke, L.; O’Hara, K.; Dixon, D. Gamification: Using game-design elements in non-gaming contexts. In Proceedings of the International Conference on Human Factors in Computing Systems (CHI 2011), Extended Abstracts Volume, Vancouver, BC, Canada, 7–12 May 2011; pp. 2425–2428. [Google Scholar]
  6. Hamari, J.; Koivisto, J.; Sarsa, H. Does Gamification Work?—A Literature Review of Empirical Studies on Gamification. In Proceedings of the 47th Hawaii International Conference on System Sciences, Waikoloa, HI, USA, 6–9 January 2014; pp. 3025–3034. [Google Scholar]
  7. Bozkurt, A.; Durak, G. A systematic review of gamification research: In pursuit of homo ludens. Int. J. Game-Based Learn. 2018, 8, 15–33. [Google Scholar] [CrossRef]
  8. Di Dio, S.; Peri, G.; Rizzo, G.; Vinci, I. Design, technology and social innovation: the serious game of Traffic O2. In Participatory Design Theory: Using Technology and Social Media to Foster Civic Engagement; Routledge: London, UK; New York, NY, USA, 2019; pp. 145–156. ISBN 9781138087682. [Google Scholar]
  9. Groos, K. The Play of Man; D. Appleton and Company: New York, NY, USA, 1901. [Google Scholar]
  10. Suits, B.H. The Grasshopper: Games, Life and Utopia, 3rd ed.; Broadview Press Ltd.: Peterborough, ON, Canada, 2005. [Google Scholar]
  11. Csikszentmihalyi, M. Flow: The Psychology of Optimal Experience, 1st ed.; HarperCollins Publishers Inc.: New York, NY, USA, 2008. [Google Scholar]
  12. MUV Mobility Urban Values. Available online: (accessed on 9 April 2019).
  13. Lissandrello, E.; Morelli, N.; Schillaci, D.; Di Dio, S. Urban innovation through co–design scenarios. In Conference on Smart Learning Ecosystems and Regional Development; Springer: Cham, Switzerland, 2018; pp. 110–122. [Google Scholar]
  14. Innes, J.E.; Booher, D.E. Reframing public participation: strategies for the 21st century. Plan. Theory Pract. 2004, 5, 419–436. [Google Scholar] [CrossRef]
  15. Manzini, E. Making things happen: Social innovation and design. Des. Issues 2014, 30, 57–66. [Google Scholar] [CrossRef]
  16. Tosey, P.; Visser, M.; Saunders, M. The origins and conceptualizations of ‘triple–loop’ learning: A critical review. Manag. Learn. 2012, 43, 291–307. [Google Scholar] [CrossRef]
  17. Nabatchi, T.; Leighninger, M. Public Participation for 21st Century Democracy; Wiley: Hoboken, NJ, USA, 2015. [Google Scholar]
  18. Di Dio, S.; Lissandrello, E.; Schillaci, D.; Caroleo, B.; Vesco, A.; D’Hespeel, I. MUV: A game to encourage sustainable mobility habits. In Lecture Notes in Computer Science Vol. 11385, Proceedings of the Games and Learning Alliance Conference (GALA 2018), Palermo, Italy, 5–7 December 2018; Gentile, M., Allegra, M., Söbke, H., Eds.; Springer: Cham, Switzerland, 2019; pp. 60–70. [Google Scholar]
  19. CIVITAS SATELLITE D2.3—Refined CIVITAS Process and Impact Evaluation Framework (31.08.2017). Available online: (accessed on 9 April 2019).
  20. European Commission. Impact Assessment Guidelines (2009). Available online: (accessed on 9 April 2019).
  21. Joumard, R.; Gudmundsson, H.; Folkeson, L. Framework for Assessing Indicators of Environmental Impacts in the Transport Sector. Trans. Res. Rec. 2011, 2242, 55–63. [Google Scholar] [CrossRef] [Green Version]
  22. Bosch, P.; Jongeneel, S.; Rovers, V.; Neumann, H.M.; Airaksinen, M.; Huovila, A. CITYkeys Indicators for Smart City Projects and Smart Cities (2017). Available online: (accessed on 10 April 2019).
  23. Ahvenniemi, H.; Huovila, A.; Pinto–Seppä, I.; Airaksinen, M. What are the differences between sustainable and smart cities? Cities 2017, 60, 234–245. [Google Scholar] [CrossRef]
  24. TrafficO2. Available online: (accessed on 10 April 2019).
  25. Evaluation Unit DEVCO. Evaluation Methodological Approach (2015). Available online: (accessed on 10 April 2019).
  26. European Environment Agency. EMEP/EEA Air Pollutant Emission Inventory Guidebook 2016: Technical Guidance to Prepare National Emission Inventories (EEA Report No 21/2016); Publications Office of the European Union: Luxembourg, 2016. [Google Scholar]
Figure 1. Screenshots of Mobility Urban Values (MUV) mobile app.
Figure 1. Screenshots of Mobility Urban Values (MUV) mobile app.
Systems 07 00030 g001
Figure 2. Before-and-after comparison envisaged by Mobility Urban Values (MUV) impact evaluation.
Figure 2. Before-and-after comparison envisaged by Mobility Urban Values (MUV) impact evaluation.
Systems 07 00030 g002
Figure 3. Data sources of Mobility Urban Values (MUV) impact indicators.
Figure 3. Data sources of Mobility Urban Values (MUV) impact indicators.
Systems 07 00030 g003
Figure 4. Process evaluation: illustrative template table to be filled in by each pilot manager for each monitoring period. MUV: Mobility Urban Values.
Figure 4. Process evaluation: illustrative template table to be filled in by each pilot manager for each monitoring period. MUV: Mobility Urban Values.
Systems 07 00030 g004
Figure 5. Average available income and unemployment rate: comparison in the six neighborhoods.
Figure 5. Average available income and unemployment rate: comparison in the six neighborhoods.
Systems 07 00030 g005
Figure 6. Smartphone ownership and Internet use on the move by age group: comparison in the six neighborhoods.
Figure 6. Smartphone ownership and Internet use on the move by age group: comparison in the six neighborhoods.
Systems 07 00030 g006
Figure 7. Gasoline prices with respect to the average available income ratio: comparison in the six neighborhoods.
Figure 7. Gasoline prices with respect to the average available income ratio: comparison in the six neighborhoods.
Systems 07 00030 g007
Figure 8. People killed in road accidents: comparison in the six neighborhoods.
Figure 8. People killed in road accidents: comparison in the six neighborhoods.
Systems 07 00030 g008
Figure 9. Weather conditions: comparison in the six neighborhoods.
Figure 9. Weather conditions: comparison in the six neighborhoods.
Systems 07 00030 g009
Figure 10. Comparison of a modal split in Palermo: data provided by the municipality vs. Mobility Urban Values (MUV) app.
Figure 10. Comparison of a modal split in Palermo: data provided by the municipality vs. Mobility Urban Values (MUV) app.
Systems 07 00030 g010
Figure 11. Preliminary shortcomings of Mobility Urban Values (MUV) process evaluation.
Figure 11. Preliminary shortcomings of Mobility Urban Values (MUV) process evaluation.
Systems 07 00030 g011
Table 1. Possible options for Mobility Urban Values (MUV) impact indicators counterfactual estimate.
Table 1. Possible options for Mobility Urban Values (MUV) impact indicators counterfactual estimate.
LabelType of Counterfactual Estimate
AConstructing a reference scenario together with the involved stakeholders (e.g., pilot managers, mobility experts, local decision makers).
BUsing the pre-MUV values at the baseline (t0) to estimate the post-MUV outcomes (counterfeit counterfactual).
CUsing as reference scenario the travel behavior’s information provided by the player during the registration of each frequent route (not necessarily at t0).
Table 2. List of Mobility Urban Values (MUV) impact indicators.
Table 2. List of Mobility Urban Values (MUV) impact indicators.
Impact Sub-AreaCodeIndicator
IA1 Society-PeopleIA1.S1 Citizens participationIA1.S1.1Awareness level
IA1.S1.2Involvement level
IA1.S1.3Acceptance level
IA1.S1.4Activeness level
IA1.S1.5Perseverance level
IA1.S2 Behavioral changeIA1.S2.1Sustainable mobility habits
IA1.S2.2Use of private car
IA1.S2.3Modal split
IA1.S2.4Travel time
IA1.S3 CommunityIA1.S3.1Community cohesion among travelers
IA1.S4 Health and wellbeingIA1.S4.1Physical activity
IA2 Society-GovernanceIA2.S1 PlanningIA2.S1.1Planning process
IA2.S2 GovernanceIA2.S1.2Quality of policies, plans, and programs
IA2.S2.1Rules and regulations
IA2.S2.3Policy making process
IA2.S2.5Cooperation structures with stakeholders
IA2.S3 Open dataIA2.S3.1Quality of open data
IA2.S3.2Open datasets
IA3 EconomyIA3.S1 Business networkIA3.S1.1Global sponsors involvement
IA3.S1.2Community interaction with global sponsors
IA3.S1.3Local sponsors involvement
IA3.S1.4Community interaction with local sponsors
IA3.S2 InvestmentsIA3.S2.1Public investments
IA3.S2.2Private investments
IA3.S3 InnovationIA3.S3.1Innovative environment
IA3.S3.2Economic activity
IA3.S3.3Open data exploitation
IA4 EnvironmentIA4.S1 Climate change (GHG)IA4.S1.1CO2 emissions from road traffic
IA4.S1.2CO2 level
IA4.S2 Pollution (emissions/noise)IA4.S2.1Noise level
IA4.S2.2NOx emissions from road traffic
IA4.S2.3NO2 level
IA4.S2.4PM2.5 1 emissions from road traffic
IA4.S2.5PM2.5 1 concentration
IA4.S2.6PM10 1 concentration
IA4.S2.7CO emissions from road traffic
IA4.S2.8CO level
IA4.S3 EnergyIA4.S3.1Energy consumption from road traffic
1 Particulate matter, with fine particles having a diameter of 2.5 μm or less (PM2.5) or with coarse particles having a diameter between 2.5 and 10 μm (PM10).
Table 3. List of context indicators used in the six neighborhoods (values are presented in Table A2). PDA = Personal Digital Assistant.
Table 3. List of context indicators used in the six neighborhoods (values are presented in Table A2). PDA = Personal Digital Assistant.
Area.CodeContext IndicatorDescription/Sub-Indicators
C2Neighborhood namename of the neighborhood
C3Area dimensionarea of the neighborhood [km2]
C4Population of the neighborhoodnumber of inhabitants in the neighborhood
C5Population of the citynumber of inhabitants in the city (municipality)
C6Population density in the neighborhoodinhabitants of the neighborhood/km2
C7Age structureage of residents in the neighborhood (share of the population in four age groups: 0–14; 15–64; 65–74; 75+)
C8Age structureage of residents in the city (share of the population in four age groups: 0–14; 15–64; 65–74; 75+)
C9Average available income (yearly)disposable income pro capita of private households by NUTS 2 regions expressed in PPS (Purchasing Power Standard) per inhabitant [tgs00026]
C10Driving age populationnumber of inhabitants with the minimum age for driving a car without supervision
C11Business densitynumber of registered firms per 1000 active people (those aged 15–64)
C12Landmarks and historic featuresHistoric point of interests in the target area (e.g., monuments, museums, ancient churches)
C13Shopping areas/commercial centersthe concentration of shopping and service centers in the area
C14Neighborhood assetsrecreation services (e.g., community centers), recreation/culture availability, cultural facilities (e.g., libraries), schools, hospitals, etc.
C15Employment rateinhabitants employed/working age population (15–64) [lfst_r_lfe2emprc]
C16Unemployment rateinhabitants unemployed/labour force
unemployment rate by NUTS 2 regions—from 15 to 74 years [lfst_r_lfu3rt]
C17Smartphone ownership% of smartphone users in the target area
C18Internet use on the move (total)% of internet users on the move (use of mobile devices via mobile or wireless connection: mobile phone (or smartphone), portable computer (e.g., laptop, tablet) or another mobile device (e.g., PDA, e-book reader) away from home or work). Base: individuals aged 16 to 74 [tin00083]
C19Internet use on the move (by age group)% of internet users on the move by age group (16–24; 25–54; 55–74)
TRANSPORTC20Public transportpublic transport lines (number of bus-tram lines available)
C21Public transportpublic transport stops (number of bus-tram stops available)
C22Public transportunderground lines (number of underground lines available)
C23Public transportunderground stops (number of underground stops available)
C24Public transportrailway stations (number of railway stations available)
C25Public transportwaiting time: average waiting time at a bus-tram station [min]
C26Public transportaverage number of daily users of public transport
C27Public transportaverage bus occupancy (average number of passengers on a bus on a working day)
C28Public transportbus fleet by fuel type: share of urban bus types operating in the city (% of diesel, CNG, electric buses)
C29Public transportaccessibility: share of the population with appropriate access to mobility services (Population living within 500 m of a bus or metro stop)
C30Bikecycling paths: length of cycling path [km]
C31Bikecycling paths (space devoted to cycling as a proportion of the target area, assuming a width of 1.5 m)
C32Bikebike racks (# bike racks per 1000 inhabitants)
C33Bikebike ownership (bikes owned per 1000 inhabitants)
C34Walkpedestrian areas:
– length of pedestrian streets [km]
– area of pedestrian squares [km2]
C35Walkpedestrian areas (space devoted to the pedestrian as the proportion of the target area). For pedestrian streets (given the street length), it is supposed a 4 m width.
C36Private carcar ownership (passenger cars per 1000 inhabitants)
C37Private caraverage car occupancy
C38Shared servicestaxi: number of taxi vehicles operating
C39Shared servicesother on-demand ridesharing services (e.g., Uber)
C40Shared servicescarpooling services (e.g., BlaBlaCar)
C41Shared servicescar sharing cars (car sharing fleet size per 1000 driving age people)
C42Shared servicesbike sharing bikes and stations (number of bike sharing bikes per 1000 inhabitants)
C43Transport costscost of a combined monthly ticket (all modes of public transport) [€]
C44Transport costsgasoline prices (June 2018) [€/l]
C45Transport costsaverage parking cost (hourly) in the neighborhood
C46Traffic congestionpeak hours (the peak hours during the weekday)
C47Traffic congestioncongestion level (increase in non-highways travel times when compared to a Free Flow situation)
C48Traffic congestionmorning peak (increase in morning peak travel times when compared to a Free Flow situation)
C49Traffic congestionevening peak (increase in evening peak travel times when compared to a Free Flow situation)
C50Traffic congestionparking: number of on-street parking spaces (on or along the curb of streets, NO parking garages)
C51Traffic congestionparking: number of off-street parking garages/parking lots and the aggregate number of parking spaces (both indoor and outdoor parking facilities)
C52Traffic congestionaverage time to travel 1 km by car on a working day [minutes]
C53Traffic congestionaverage time to travel 1 km by bus on a working day [minutes]
C54Traffic congestionaverage time to travel 1 km by moto on a working day [minutes]
C55Safety and securitypeople killed in road accidents per 10,000 population [urb_ctran]
C56Safety and securitynumber of car accidents (last year available) car/car
C57Safety and securitynumber of accidents involving a bike (last year available)—car/bike
C58Safety and securitynumber of accidents involving pedestrians (last year available)—car/people
C59Safety and securitynumber of accidents involving a bus (last year available)
ENV.C60Weather conditionsaverage number of days with precipitation (yearly)
C61Weather conditionsaverage annual temperature
INSTITUTIONALC62Sustainable urban mobility plan (SUMP)does a SUMP exist?
C63Easementsmobility subsidies (e.g., for electric vehicles, discounts on parking)
C64Mobility policiesfor example, limited traffic zones, priority for pedestrians, etc.
C65Mobility initiatives/infrastructure already plannedin the neighborhood
C66Mobility initiatives/infrastructure already plannedin other parts of the city

Share and Cite

MDPI and ACS Style

Caroleo, B.; Morelli, N.; Lissandrello, E.; Vesco, A.; Di Dio, S.; Mauro, S. Measuring the Change Towards More Sustainable Mobility: MUV Impact Evaluation Approach. Systems 2019, 7, 30.

AMA Style

Caroleo B, Morelli N, Lissandrello E, Vesco A, Di Dio S, Mauro S. Measuring the Change Towards More Sustainable Mobility: MUV Impact Evaluation Approach. Systems. 2019; 7(2):30.

Chicago/Turabian Style

Caroleo, Brunella, Nicola Morelli, Enza Lissandrello, Andrea Vesco, Salvatore Di Dio, and Stefania Mauro. 2019. "Measuring the Change Towards More Sustainable Mobility: MUV Impact Evaluation Approach" Systems 7, no. 2: 30.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop