Next Article in Journal
Colonialism on Spatial Transformation: A Socio-Spatial Analysis of the Outdoor Transitional Spaces of Sri Lankan House
Previous Article in Journal
Financing Sustainable Energy Efficiency Projects: The Triple-A Case
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Proceeding Paper

Sustainable Cities: A KPI-Driven Sustainable Evaluation Framework for Smart Cities †

Foundation CARTIF, Parque Tecnológico Boecillo 205, 47151 Boecillo, Spain
CEREMA, Direction Territoriale Ouest, MAN; rue René Viviani, 44000 Nantes, France
VTT Technical Research Centre of Finland Ltd., P.O. Box 1000, 02044 Espoo, Finland
TECNALIA, Parque Científico y Tecnológico de Bizkaia 700, 48160 Derio, Spain
Center for Public Governance (ESADEgov), Universitat Ramon Llull—ESADE, Pedralbes, 60, 62, 08034 Barcelona, Spain
Departament of Project and Construction Engineering, Universitat Politècnica de Catalunya (BarcelonaTech), Jordi Girona 1-3, 08034 Barcelona, Spain
Deparment of Urban Planning, HafenCity University Hamburg, Henning-Voscherau-Platz 1, 20457 Hamburg, Germany
Author to whom correspondence should be addressed.
Presented at the Sustainable Places 2021, Rome, Italy, 29 September–1 October 2021.
Environ. Sci. Proc. 2021, 11(1), 21;
Published: 1 December 2021
(This article belongs to the Proceedings of The 9th Annual Edition of Sustainable Places (SP 2021))


Cities are currently dealing with the sustainable transition from carbon-based economies to carbon-neutral cities. The H2020 mySMARTLife project seeks to demonstrate innovative technologies through the implementation of about 150 innovative energy solutions in the cites of Nantes, Hamburg and Helsinki. The evaluation framework that was defined and validated in the project enables the assessment of the environmental, energy, economic, social, ICT and governance aspects of replicable and sustainable smart city solutions with a high market potential. The main features and the process for their definition will be described in this paper.

1. Introduction

Nowadays, cities have become hubs for modern civilizations [1], which has implications for the need for more natural resources. The limited nature of these resources increases the importance of the sustainable transformation of cities by the exploration of new technologies [2] that support urban transformation. To this end, the EU has adopted ambitious legislation in order to implement its international commitments to climate change by establishing a 55% reduction in GHG emissions by 2030 and climate neutrality by 2050 [3]. The main goal is the decarbonization of European cities and the penetration of renewable energy sources.
By 2050, it is estimated that 85% of European people will be living in urban areas [4]. By that time, new concepts for urban transformation such as Smart Energy, Sustainable Mobility, Smart People and Economy will be required, all of which would be supported by the integration of ICTs (Information and Communication Technologies) [5].
Under this perspective, the H2020-funded project mySMARTLife (Grant Agreement #731297) [6] contributes through the sustainable transformation of the three lighthouse cities of Nantes, Hamburg and Helsinki. Within the project, more than 150 actions have been deployed in the pillars of energy, mobility, ICT, citizens, economy and governance. With all of these aspects in mind, the target of mySMARTLife is to achieve a renewable share of 54%, while reducing 55% of the greenhouse emissions that are due to buildings and transportation actions. In addition, it fosters the growth of the local economy and entrepreneurship and the deployment of integrated urban planning in order to prioritize the most adequate actions for covering the city needs.
These targets required a rigorous assessment plan. To that end, this paper presents an evaluation framework, which is driven by KPIs (Key Performance Indicators), so as to provide cities with the methodologies to quantify the real achieved impacts. The proposed framework covers both the city and project levels (understanding the project as the area of execution of the aforementioned actions).

2. Research Methods and Background

The development of the evaluation framework was the result of collaboration among research centres, technology providers and cities, thus creating a co-creation strategy between stakeholders. The definition of this framework followed a research methodology that consisted of four main phases:
establishing the objectives of the cities for sustainability and Smart Cities urban plans.
search for bibliography, standards and previous works.
identification of the suitable indicators for Smart and Sustainable Cities assessment.
definition of the KPI-driven evaluation framework combining the objectives and indicators for the multiple city domains.
Within the research methodology, the analysis of previous studies is crucial. In this sense, the authors in [1] highlight the importance of making use of indicators for the better management of cities. In particular, they propose performance indicators in the areas of the economy, the environment, society, governance, energy, infrastructure, transportation, and pandemic resiliency. However, indicators are not enough for proper assessment. They provide the quantitative calculations, but the following of standard protocols for the evaluation is also required, for instance, the IPMVP (International Performance Measurement and Verification Protocol). Moreover, they are neglecting the digitalization of cities in the form of ICT actions, which are also very important in Smart City transformation, as explained by [5].
Authors in [7] proposed an assessment strategy that mainly considered building and district renovation, urban mobility and ICTs, while social and governance aspects were not fully addressed. Nevertheless, as introduced, citizens should be at the core of the urban transformation, and their involvement, acceptance and engagement are crucial. Finally, the research in [8] proposed a framework where the levels of Smart Economy, Smart Mobility, Smart Environment, Smart People, Smart Living and Smart Governance were integrated. However, the support of ICTs was again neglected.
With these studies in mind, the mySMARTLife KPI-driven evaluation framework goes a step forward by proposing an integrated methodology where Smart Energy, Smart Mobility, City Infrastructures (including ICTs), Smart People, Smart Economy and Smart Governance are combined. This framework is supported by a set of KPIs to quantify the impacts. In addition, it provides a two-scale analysis of the city, i.e., it evaluates the actions and extrapolates the results to the city level.

3. mySMARTLife KPI-Driven Evaluation Framework

This section describes the aforementioned evaluation framework. Figure 1 [9] illustrates the proposed framework. First of all, it should be highlighted that two levels of assessment are included:
  • Project level includes the more than 150 actions that are being deployed in the specific areas of the cities that are involved in the project. The main aim is to obtain the quantitative analysis of the impacts of those actions (e.g., building retrofitting, integration of renewables, electrification of the transport, etc.) as well as the performance of the technological solutions.
  • City level, which extrapolates the quantified impacts from the executed project in order to estimate the impact that these actions would have on the city. The outcome of this level is to support cities at the time of planning of urban transformation strategies by following quantitative and objective methods that are driven by KPIs.
Within each of the stated levels, a set of categories (named fields in the Smart City vision and pillars for the project level) is established. These are, as mentioned above, energy and environment, mobility, urban infrastructures (including the digitalization of the city through the ICTs and urban data platform deployment), citizens, economy and governance. All of them are driven by a set of indicators [9], which are summarized in Table 1.
Finally, the framework complements the indicators and definitions that are derived from the methodologies and protocols, with the aim of supporting cities during the implementation of the project evaluation framework. These protocols should be just considered as a guidance as there are other suitable protocols to be applied. Thus, the evaluation framework does not only provide a theoretical indicator-based procedure, but also pathways to apply them to analyse the success of the implemented actions. Table 2 depicts a summary of the proposed methodologies for each one of the project pillars.

4. Discussion and Conclusions

This paper has presented a proposed KPI-driven evaluation framework in order to allow cities to evaluate the achieved impacts by the deployment of sustainable actions. In this sense, this framework has been applied in the three lighthouse cities of Nantes, Hamburg and Helsinki. In the first stage of the project, the framework and their indicators have been used for obtaining the city audits, as well as determining the baselines for the mySMARTLife specific actions. Thanks to these common KPIs and methods, comparative results may be obtained. That is to say, for instance, in the energy field, there is a dependency on climate conditions to be further adjusted in the models in order to compare different periods with diverse climate conditions. These are routine adjustments according to the IPMVP, which are sometimes neglected and consequently can lead to wrong assumptions, resulting in a non-realistic or distorted view of the impacts.
The applicability of the framework relies on the selection of the most suitable KPIs for each of the cities. Although it provides a wide set of indicators, not all of them are applicable or quantifiable; therefore, cities should first identify the ones that are aligned with the action targets or Smart City urban plans. Nevertheless, it is a benefit for the cities in terms of having a broad, flexible, and replicable methodology for the evaluation of the impacts in terms of sustainability.
MySMARTLife has recently finished the implementation of the actions and, nowadays, is starting the final stage of the monitoring period and KPI calculations. In this sense, future work could investigate the application of the framework and indicators for the reporting period (i.e., the period after the interventions) as the next step. As stated before, the following of common protocols and methodologies allows the adjustments between periods to provide real achievements and/or impacts. Additionally, it should be noted that the evaluation framework is a valuable tool to support cities in their urban transformation processes by providing not only the path to evaluate the performance and impacts of innovative energy-efficient solutions from environmental and energy points of view, but also the economic viability, the social acceptance and the enhancement of city governance, which are in most cases the main barriers that prevent cities from advancing towards sustainability and climate neutrality.

Data Availability Statement


The authors would like to thank, first, the EC for funding and supporting mySMARTLife project under GA#731297. Secondly, the authors would like to thank the rest of consortium of mySMARTLife for the contributions in the definition of the framework.


  1. Abu-Rayash, A.; Dincer, I. Development of integrated sustainability performance indicators for better management of smart cities. Sustain. Cities Soc. 2021, 67, 102704. [Google Scholar] [CrossRef]
  2. Akande, A.; Cabral, P.; Casteleyn, S. Understanding the sharing economy and its implication on sustainability in smart cities. J. Clean. Prod. 2020, 277, 124077. [Google Scholar] [CrossRef]
  3. 2030 Climate & Energy Framework. Available online: (accessed on 25 September 2021).
  4. EU Research and Innovation; Final Report of the High-Level Panel of the European Decarbonisation Pathways Initiative; European Commission: Brussels, Belgium, 2018.
  5. Bibri, S.E. Transitioning from Smart Cities to Smarter Cities: The Future Potential of ICT of Pervasive Computing for Advancing Environmental Sustainability. In The Urban Book Series: Smart Sustainable Cities of the Future; Springer: Berlin, Germany, 2018. [Google Scholar]
  6. mySMARTLife Project: GA #731297. Available online: (accessed on 19 September 2021).
  7. García-Fuentes, M.Á.; Antolín, J.; de Torre, C.; Pérez, A.; Tomé, I.; Mirantes, M.L.; López, F.; Martín, J.; Gómez, J. Evaluation of Results of City Sustainable Transformation Projects in the Fields of Mobility and Energy Efficiency with Real Application in a District in Valladolid (Spain). Sustainability 2021, 13, 9683. [Google Scholar] [CrossRef]
  8. Herrera, F.; Fajardo, C. A framework for measuring smart cities. In Proceedings of the 15th Annual International Conference on Digital Government Research, New York, NY, USA, 18–21 June 2014; pp. 44–54. [Google Scholar] [CrossRef]
  9. mySMARTLife Partners. Integrated Evaluation Procedure; Technical Report; mySMARTLife: Boecillo, Spain, 2019. [Google Scholar]
Figure 1. mySMARTLife KPI-evaluation framework [9].
Figure 1. mySMARTLife KPI-evaluation framework [9].
Environsciproc 11 00021 g001
Table 1. Number of indicators defined per pillar.
Table 1. Number of indicators defined per pillar.
Core CategoriesCity LevelProject Level
Energy & Environment5632
Urban infrastructures/ICT & Urban platform2011
Table 2. Evaluation methods for each one of the categories.
Table 2. Evaluation methods for each one of the categories.
Core CategoriesEvaluation Methodology
Energy & EnvironmentExtension of IPVMP
MobilityCO2 emissions-based
Urban infrastructureSoftware metrics
CitizensSurveys / Interviews
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Quijano, A.; Hernández, J.L.; Nouaille, P.; Virtanen, M.; Sánchez-Sarachu, B.; Pardo-Bosch, F.; Knieilng, J. Sustainable Cities: A KPI-Driven Sustainable Evaluation Framework for Smart Cities. Environ. Sci. Proc. 2021, 11, 21.

AMA Style

Quijano A, Hernández JL, Nouaille P, Virtanen M, Sánchez-Sarachu B, Pardo-Bosch F, Knieilng J. Sustainable Cities: A KPI-Driven Sustainable Evaluation Framework for Smart Cities. Environmental Sciences Proceedings. 2021; 11(1):21.

Chicago/Turabian Style

Quijano, Ana, Jose L. Hernández, Pierre Nouaille, Mikko Virtanen, Beatriz Sánchez-Sarachu, Francesc Pardo-Bosch, and Jörg Knieilng. 2021. "Sustainable Cities: A KPI-Driven Sustainable Evaluation Framework for Smart Cities" Environmental Sciences Proceedings 11, no. 1: 21.

Article Metrics

Back to TopTop