Evaluating Smart and Sustainable City Projects: An Integrated Framework of Impact and Performance Indicators

Highlights

What are the main findings?

• A comprehensive framework of indicators has been developed for project-level evaluation in smart cities, based on the analysis of 14 international systems and more than 1200 indicators classified in a unified taxonomy.
• The framework defines 73 project evaluation areas and develops impact indicators linked to performance metrics, ensuring a holistic and practical approach to project assessment.

What are the implication of the main findings?

• The framework addresses a gap in project-level evaluation tools by providing a balanced system that overcomes redundancies and thematic imbalances in existing indicator frameworks.
• It offers urban planners and policymakers, especially in small and medium-sized cities, a practical tool for evidence-based decision-making, bridging the gap between abstract smart city strategies and tangible project outcomes.

Abstract

Smart and sustainable cities are often assessed using indicator-based models. However, most existing systems evaluate cities as a whole, offering limited support for project-level decision-making, particularly in small and medium-sized cities with scarce resources. This study aims to fill this gap by developing a comprehensive indicator framework tailored to the evaluation of smart city projects, designed to guide investment choices and support evidence-based planning. To build this framework, a systematic review of international indicator systems was conducted, compiling and refining over 1200 indicators into a unified taxonomy. The analysis revealed structural imbalances, with environmental and social dimensions prevailing over economic and governance aspects, and confirmed substantial redundancies, with nearly one-third of indicators overlapping. Using project actions as an analytical lens, gaps were detected and 73 evaluation areas defined. From these, anticipated impact indicators were developed and linked to corresponding performance metrics. Beyond consolidating fragmented systems, the framework provides a practical and balanced tool for multidimensional project assessment. An initial empirical pre-validation demonstrated its coverage and usability, reinforcing its potential to support planners and policymakers in comparing investment alternatives. Unlike traditional ranking or maturity models, it directly bridges the gap between abstract smart city strategies and tangible, project-level outcomes.

1. Introduction

The Smart and Sustainable City paradigm has been widely adopted by cities as a model for urban transformation in response to the contemporary context they face [1]. This context is shaped, on the one hand, by the sustainability agenda cities are expected to meet, crystallized in the United Nations’ Sustainable Development Goals, in particular Goal 11, “make cities and human settlements inclusive, safe, resilient and sustainable” [2], and, on the other hand, by the new urban morphology driven by the emergence and adoption of digital technologies, whereby cities function as cyber–physical–social spaces in which these three components interact intensively and in complex ways [3,4]. Leaving aside the theoretical debates around the concept of the Smart and Sustainable City, including its multiple definitions in the academic literature [5], from a practical perspective, it can be seen as the adaptation of traditional urban planning to this new and more complex context [6].

Among the generally accepted characteristics of Smart Cities, such as their holistic nature [7] and the importance of involving urban stakeholders and citizens [8,9], measurability stands out as a core component, represented by the second element of the SMART acronym, “measurable” [10,11,12]. Cities rely on evaluation models developed within the scientific community, based on sets of indicators covering different urban dimensions, as tools to measure progress in the transformation process implied by the Smart and Sustainable City [13]. These evaluation models present different typologies, the most common being those that assess the city as a whole, measuring overall performance, maturity, or producing rankings [14,15,16,17,18]. Such frameworks address questions about the degree of transformation achieved or make a comparison with other cities. While these models work well for consolidated processes in large cities, they are of limited use for policymakers’ decision-making [19], particularly in smaller cities, which require more specific tools [20] oriented toward supporting investment choices [21]. In these cities, the need to consider the context [13,22] and translate results into actionable guidance [23] becomes critical [24]. However, widely used city-level frameworks are often too extensive and benchmarking-oriented. Smaller cities also play a significant role in achieving global sustainability goals [25], thereby reinforcing the relevance of their consideration. Yet, specific models developed in the literature for small cities remain scarce and fragmented; examples include ref. [20] as a general evaluation model, ref. [24] as a maturity analysis, and ref. [21] as a readiness assessment tool.

Although numerous evaluation models have been developed at the city level, project-level assessment remains largely underexplored, with only a few dedicated examples available [26,27,28], even though these projects are the concrete instruments through which smart city policies are implemented in practice [10,29]. In contrast to general indicator frameworks, these models address a more operational question: the extent to which a specific project contributes to the process of urban transformation [26], which makes philosophy and methods of evaluation very different. As mentioned, most existing indicator systems were originally designed to measure overall city performance or maturity, rather than to guide investment choices for specific initiatives [30]. These frameworks typically lack the level of detail needed to evaluate individual actions. In the context of high opportunity costs in choosing between investment alternatives and given the fundamental role of indicator systems in diagnosing and promoting smart and sustainable city development [31], such tools represent essential support for decision-making [22]. Their relevance is even greater for small cities [21], where the need to evaluate the effects of planned initiatives is especially pressing [24,32].

The main objective of this study is to establish a comprehensive collection of indicators that enable the assessment of the anticipated impact of smart city projects across the different urban dimensions during the planning, decision-making, and investment stages, as these are the critical stages where project selection and prioritization occur, and where reliable assessment tools to support evidence-based investment choices are particularly lacking. Complementarily, a collection of post-performance indicators is established, linked to each expected impact indicator, allowing for the evaluation of projects during implementation, monitoring and post-implementation stages. The framework is oriented primarily toward planners and urban managers in cities where smart city policies are not yet consolidated or where resources are scarce, as is often the case in small and medium-sized cities.

The contribution of this study lies in addressing the general gap observed in evaluation models at the project level for smart and sustainable cities, a gap particularly evident in indicator collections designed to support decision-making and investment choices within smart city policies. The indicator framework developed here is grounded in the most relevant systems in the academic literature, selected to form a representative sample that includes different perspectives on smart cities. In this way, the logic of city-wide indicator systems is adapted to the project scale. Based on this theoretical foundation, a multi-stage methodology was designed to produce the final framework, overcoming the limitations and deficiencies identified in current systems and meeting a set of key requirements identified in the literature:

• The need for impact indicators that measure the contribution of projects and initiatives to intended objectives [22,33].
• The importance of indicators designed to inform decision-making [11,28,34].
• There is no clear consensus on how to translate the results of evaluations into decision-making [35].
• The development of specific frameworks combining multiple perspectives on smart cities and integrating both quantitative/specific and qualitative indicators [23].
• The need for a holistic vision of the city [13] and an integrated perspective to ensure effectiveness [31].
• The challenge of simplicity is providing city managers and planning officers with tools that avoid inefficiencies in the evaluation process [22,23].
• The need for a clear distribution and structure of indicators across dimensions and subdimensions, with a balanced and representative set that prioritizes quality over quantity [13].

Beyond the construction of the framework itself, this study makes an additional contribution by analyzing existing collections of smart city indicators. The results reveal not only the extent of redundancies across systems, but also structural imbalances in their thematic coverage. By evidencing these inconsistencies in a comparative and comprehensive manner, the research provides novel insights into the current state of indicator systems and reinforces the rationale for developing a refined, balanced, and project-oriented framework. Finally, the developed framework undergoes a testing process as a form of empirical pre-validation through its application to a set of projects in the context of a city with a smart cities program as a case study, analyzing the results, the process of using the model and comparing the results with the application of other indicator systems.

The article is structured into five main sections: this introductory part, followed by a background section that provides essential context on indicator systems by reviewing their evolution and defining characteristics. Then, the third section includes a description of the methodological stages used to develop the framework, and, finally, the fourth and fifth sections present the results, discussion, and conclusions of the research.

2. Background

2.1. The Smart, Sustainable and Measurable City

The need for indicators to measure the performance of the smart and sustainable city is consistently emphasized in the scientific literature: as an inherent characteristic of the very concept [12], as a tool to monitor, understand, analyze, and plan the smart city [11], to measure the contribution of initiatives to sustainability objectives [33], and for their fundamental importance in decision-making [11,22,28].

Over the past decades, there has been a proliferation of evaluation models that use indicators as the basis for assessing smart cities [35]. One model with significant influence on subsequent frameworks is that of ref. [16], which introduced, for the first time, a six-dimensional structure to reflect the holistic nature of the smart city, around which a collection of indicators was organized. This structure has decisively shaped most subsequent models, whether focused on rankings [36], general performance measurement of smart cities [7], or assessments of maturity levels [37]. The development of evaluation models and indicator systems has been driven not only by the academic community but also by institutional and corporate actors, with different scopes and geographic levels of analysis [13]. In recent years, several literature reviews have examined these evaluation models in depth [23,38,39,40,41], alongside studies that have analyzed in detail the components of selected indicator frameworks, whether from a general perspective or focused on specific thematic aspects [13,22,34,42,43].

The ISO 37120 standard, first published in 2014 and updated in 2018 [44], along with its complementary standards ISO 37122 [45] and ISO 37123 [46], released in 2019, seeks to standardize collections of indicators for measuring the performance of smart and sustainable cities. These address different domains such as urban services and quality of life, smart cities, and resilient cities [44,45,46]. Their influence on subsequent models is evident in the widespread adoption of their indicators, making them increasingly prominent in recent years [17,23]. Another widely adopted system of indicators is that developed by the ITU Focus Group on Smart Sustainable Cities, which produced three recommendations focused on the use of ICT, sustainability in smart cities, and alignment with the Sustainable Development Goals [47,48,49]. Finally, the Global Indicator Framework for the Sustainable Development Goals and Targets of the 2030 Agenda for Sustainable Development, adopted by the United Nations, represents an internationally recognized system of indicators for global sustainability assessment, including the specific Goal 11 on urban areas [50].

Despite the abundance of existing indicator systems, their mere availability does not solve the measurement challenges associated with smart and sustainable cities. Academic literature highlights multiple factors that condition the selection of one system over another. Among these are the absence of a universally accepted framework and the need for deep expert knowledge to identify the most appropriate option [34]. Local characteristics also play a role [17], as do city size [20], context [22], and the specific objectives of the evaluation [23]. Moreover, smart city rankings have limited usefulness for decision-making [19], especially in smaller cities, while national or regional indicator systems may be suitable for larger urban contexts but less so for smaller ones [20]. The use of commercial indicator systems has also been questioned, owing to a lack of transparency and scientific rigor [22], as these often serve primarily as instruments of marketing and symbolic prestige [19]. In some cases, the choice of a framework lacks transparency and is oriented mainly toward supporting predefined policies [34].

2.2. Indicators in Smart and Sustainable Cities

The design and use of indicators in the context of smart cities are guided by criteria intended to guarantee their validity, applicability, and usefulness both for urban management and for communication with citizens. The literature highlights that indicators fulfill three fundamental functions: quantification, simplification, and communication [22].

From a normative perspective, ISO standards specify that indicators should be comprehensive, technology-neutral, simple in interpretation, valid and verifiable through scientific methodologies, and supported by the availability of quality data to ensure consistent monitoring over time [45]. Complementarily, initiatives aimed at developing indicator frameworks, such as ref. [34], underline the importance of data accessibility and accuracy, the availability of historical series to identify trends, comparability across jurisdictions, and the independence of metrics to avoid redundancies. They also emphasize policy relevance, the potential to establish benchmark values, and the communicative appeal of indicators for citizens and the media [34].

The literature further distinguishes between objective and subjective indicators [35]. While the former measures physical and infrastructural aspects of the city, the latter captures perceptions of citizen satisfaction in areas such as the quality of education services or the transparency of administration [11]. This combination strengthens the integral vision of the smart city by complementing the technical dimension with the lived experience of its inhabitants. The CityKeys project [28] contributes an additional functional typology, differentiating between input indicators (resources employed), process indicators (activities carried out), output indicators (products generated), outcome indicators (immediate effects), and impact indicators (city-level changes). This classification facilitates the evaluation of urban projects across all stages and connects directly with the strategic objectives being pursued.

Finally, considering the growing volume of urban data, the careful selection of a reduced number of key indicators is essential to provide managers with a concise overview of city performance and to foster transparency toward citizens, often through indicator dashboards or urban scorecards [22].

3. Methods and Materials

3.1. Identification of Indicator Sources

In Section 2, the wide range of smart city evaluation models and their diverse indicator frameworks are discussed. In this first stage of the methodology, we identified the systems that would serve as the theoretical foundation for a project-level framework. To this end, we apply an extensive literature review, though indirectly with respect to the final research objective: the focus of this review is not on individual evaluation models, but rather on articles and research studies that analyze and compare multiple indicator frameworks, Although such studies are much less frequent than evaluation tools themselves, they offer a broader overview of the different smart city indicator systems in use, which provides a clear advantage for the scope of this research.

The search was conducted in the Web of Science database, restricted to research articles and reviews analyzing multiple normative frameworks or indicator collections. The strategy combined four blocks with the “AND” operator: (i) comparative approaches (“compar*”, “review”, “meta-analys*”, etc.); (ii) indicators (“indicator*”, “metric*”, “index*”); (iii) smart city context (“smart” NEAR/1 “city/cities/community”), and (iv) references to widely used frameworks (“ISO”, “ITU”, “ranking*”, “corporate”, “industry”, “standard*”). “OR” grouped synonyms and “NEAR/n” ensured semantic proximity.

After an initial screening for general relevance, 42 results remained. These were then filtered using three inclusion criteria: the article had to be published in English, within the last decade and focused specifically on the analysis of smart and sustainable cities. This process yielded a total of 14 studies, to which two additional works were added (

Table 1. List of selected works for the detection of indicator frameworks.

Reference	Scope
Sharifi, 2019 [13]	Analysis of smart city assessment tools.
Naguib & Ragheb, 2022 [33]	Framework for analyzing smart cities from a sustainability perspective.
Carli et al., 2013 [11]	Classification of smart city indicators.
Lacson et al., 2023 [23]	Review of smart city assessment.
Toh, 2022 [19]	Study of six smart city evaluation indices.
Panagiotopoulou et al., 2020 [34]	Framework for studying indicator collections.
Dall’O et al., 2017 [20]	Small Smart Cities Assessment Model.
Shen et al., 2018 [15]	Smart city assessment model in China.
Lombardi et al., 2012 [51]	Smart city performance model in the European Union.
Yigitcanlar et al., 2022 [52]	Model for identifying readiness for a smart city.
He, 2023 [43]	Review of metrics for performance assessment.
Gazzeh, 2023 [42]	Indicators analysis and collection.
Li et al., 2019 [53]	Smart City Maturity Assessment in China.
Liao et al., 2017 [31]	Comparative analysis of indicator systems.
Huovila et al., 2019 [22]	Comparative analysis of standardized indicator systems.
Ahvenniemi et al., 2017 [54]	Analysis of evaluation models for smart and sustainable cities.

) due to their relevance and influence in subsequent research, despite being published more than ten years ago.

The exhaustive analysis of the 16 selected studies identified a total of 66 indicator systems (included in Supplementary S1. A comprehensive compilation of all the indicators from these systems would result in a dataset that is excessively large, difficult to manage, and highly redundant. The final selection of indicator systems for detailed analysis was based on four criteria:

• Relevance: Frequency of use in smart city evaluation models and degree of adoption (Figure 1).
• Origin: Inclusion of standardized systems, frameworks from evaluation models, international organizations, and corporate initiatives.
• Analytical approach: Complementarity of perspectives, covering sustainability, technology, resilience, and integrative analyses.
• Geographical scope: Diversity of origins, with both regional and global coverage.

In addition, the selected systems had to contain a specific collection of indicators, that is, measurable metrics assessing various aspects of smart and sustainable cities, whether quantitative or qualitative, rather than being limited to general policy guidelines.

Based on these criteria, a total of 14 indicator systems were selected for detailed component analysis, as presented in

Table 2. Selected indicator systems.

Indicator System	Year	Acronym Used
ISO 37120: Sustainable cities and communities Indicators for city services and quality of life [44]	2018	ISO37120
ISO 37122: Sustainable cities and communities Indicators for smart cities [45]	2019	ISO37122
ISO 37123: Sustainable cities and communities Indicators for resilient cities [46]	2019	ISO37123
ITU 1601: Key performance indicators related to the use of information and communication technology in smart sustainable cities [47]	2016	ITU1601
ITU 1602: Key performance indicators related to the sustainability impacts of information and communication technology in smart sustainable cities [48]	2016	ITU1602
ITU 1603: Evaluation and assessment Key performance indicators for smart sustainable cities to assess the achievement of sustainable development goals [49]	2016	ITU1603
Smart Cities Ranking of European Medium-Sized Cities [16]	2007–2014	SCREMC
UN, Global indicator framework for the Sustainable Development Goals and targets of the 2030 Agenda for Sustainable Development [50]	2017–2024	UNG
Boyd–Cohen Smart City Index [36]	2014	SMI
Smart Cities Index—India, ISB [55]	2019	ISB
Cities in Motion Index, IESE [56]	2018–2023	IESE
China Smart City Performance [15]	2019	CSCP
CITYkeys-ETSI [28]	2017–2018	CKE
City Protocol Society [57]	2015	CPS

.

The methodology for obtaining the framework is developed based on these selected indicator systems. This consists of a series of successive stages in which intermediate results are obtained until the final definition of the framework components (Figure 2).

3.2. Taxonomy of Indicators

Each of the systems has its own internal structure for classifying indicators by thematic areas, which differ from one another. The first step in the data processing was therefore to establish a unified taxonomy capable of classifying indicators from all sources. As an initial classification, we adopted the six dimensions proposed by ref. [16], which have become a de facto standard in the current literature due to their widespread use across the majority of evaluation models and comparative indicator studies, either directly or with minor variations [12,14,36,43,58]. Given the need for a more detailed level of classification below these basic dimensions, the taxonomy was further refined using subdimensions derived from a project evaluation model specifically developed for small cities [59], the main target of this study. The complete taxonomy structure applied in this research is presented in

Table 3. Indicator classification structure.

Dimensions	Subdimensions	Acronym
ECONOMY AND COMPETITIVENESS (ECO)	Business and labor innovation	BLI
	Entrepreneurship	ENT
	Productivity	PROD
	Local–global interconnectivity	LGI
HUMAN AND INTELLECTUAL CAPITAL (HIC)	Academic and digital training	ACA
	Creativity	CREA
	Management and promotion of urban life	URL
	Work flexibility and work–life balance	WOR
GOVERNANCE (GOV)	Transparency and citizen communication channels	CCC
	E-government and online services	EGOV
	Participation in decision-making	PART
	Innovation and efficiency in municipal management	MUM
INFRASTRUCTURE AND MOBILITY (INF)	Public transport and multimodal network	PUBTR
	ICT infrastructures	ICT
	Infrastructures, traffic and urban logistics	INFRA
	Sustainable mobility	MOB
ENVIRONMENT AND ENERGY (ENV)	Energy efficiency	ENE
	Resource and waste management	RWN
	Environmental protection and monitoring	ENPR
	Renewable energy and social awareness	RENE
SOCIAL WELFARE AND SERVICES (SOW)	Public, social and security services	PUSER
	Tourism, culture and leisure	TOCUL
	Social cohesion and inclusion	SOC
	Health and welfare	HEA

. This table lists the full names of the dimensions and subdimensions, although the results section of this study uses the short names for the sake of brevity.

The indicators collected from the 14 selected systems will be classified according to this taxonomy, and the subsequent analysis of the indicators will be carried out through the subdimensions defined here, at the level of each subdimension.

3.3. Selection of Indicators

The next step was to refine the raw list by applying two successive filters:

• Filter I: Redundancy. Many systems are influenced by one another and often include very similar or identical metrics. Since the aspects to be measured in smart and sustainable cities are finite, overlaps are inevitable. This filter, therefore, removed redundancies across systems.
• Filter II: Relevance to project-level evaluation. Most indicator systems were designed to assess cities as a whole, and their indicators are not easily adaptable to project evaluation, which requires a more concrete and context-specific approach. This filter excluded indicators that were irrelevant, insufficiently precise, or unsuitable for application at the project scale. Indicators were discarded when they were too general in scope, required data available only at national or regional levels, measured magnitudes that cannot be directly influenced by project actions, or were vaguely defined in the source frameworks without a clear measurement procedure.

3.4. Introduction of Project Actions: Gap Detection and Definition of Evaluation Areas

The application of filters resulted in an initial list of indicators, organized by subdimensions, that could potentially be used for project evaluation. The next step was to verify the completeness of this list, that is, to identify potential gaps in measuring all relevant aspects of a smart and sustainable city project, and to establish the evaluation areas on which the development of project impact indicators would be based.

Since the evaluation of a project is carried out through the actions it comprises, this dual objective was addressed by cross-referencing the initial list of indicators with a sufficiently comprehensive catalog of project actions. For this purpose, we drew on the list of project actions developed in the ASCIMER [26], led by the European Investment Bank, which aimed to design a methodology for evaluating smart city projects in Mediterranean countries, both in the North and the South. The model includes a total of 133 project actions grouped into 36 categories and represents, a priori, a sufficiently comprehensive catalog of potential project types applicable to cities at different stages of development [26]. The list of project actions was then reclassified by subdimension to align with the established indicator taxonomy and to test its level of completeness and comprehensiveness.

With the results of Filter II and the reclassified set of project actions, an individual analysis was carried out within each subdimension, assigning the indicators according to the variable being measured to each project action. Coverage was then assessed: if no indicator captured a relevant variable, or if assigned indicators left material aspects unmeasured, a gap was recorded, and a new indicator was drafted. Additionally, mapping project actions to their assigned indicators enabled the detection of groups of impact indicator typologies. Therefore, this process of cross-referencing the project action list with the initial set of indicators allowed, on the one hand, the identification of gaps to be addressed in the subsequent development of new indicators, and on the other hand, the establishment of evaluation areas as the foundation for the development of project impact indicators.

3.5. Development of Performance and Expected Impact Indicators

Once the gaps in indicators for project performance evaluation have been identified, the new indicators required are defined directly by identifying the fundamental variables and metrics that need to be measured.

On the other hand, in order to establish the expected impact indicators, an analysis is carried out of the identified evaluation areas that serve as the basis for this definition. For these indicators, it is also necessary to define a set of specific guidelines in addition to the general criteria presented in Section 2.2. Indicators had to adhere to the following:

• Focus on impacts: Measuring the effects of project actions rather than means or objectives [22,28].
• Use a baseline: Comparing impacts against the pre-project situation.
• Provide estimates: Anticipating how the project will affect the city once implemented.
• Remain simple: Easily assessed and concise, serving as decision-making tools for planners, particularly in smaller cities [20,28].
• Ensure objectivity: Allowing subsequent monitoring and verification despite their anticipatory nature.
• Maintain clarity: Intuitive and unambiguous formulation and assessment methods.

Based on these guidelines and the defined evaluation areas, the final set of project impact indicators was created, each documented through an individual profile sheet that also includes the corresponding post-performance indicators.

Finally, as an empirical pre-validation, the framework was applied to assess the impact of a set of projects in the case study of the city of Alcoy, located in southeastern Spain. With a population of 60,000 inhabitants, Alcoy falls within the small-city category as defined by the EU [60]. Beyond matching the target city profile for framework application due to its size, it is also an ideal testing ground given its implementation of a smart and sustainable city program supported by a dedicated technical department, which implies a wealth of data not found in cities of this size.

4. Results and Discussion

4.1. Collection of Indicators

An initial analysis of the nature of the selected systems indicates that most of the selected indicator systems are primarily focused on the evaluation of smart cities, either with the aim of producing rankings or of assessing performance and maturity levels, rather than on the evaluation of specific projects, consistent with the gap identified in the scientific literature. An exception in this regard is the CKE framework, which provides a specific collection of indicators both for projects and for cities.

Regarding the general typology of indicators, most of the analyzed frameworks differentiate between types of indicators within city-level assessment, such as core, supporting, and profile indicators in the case of ISO standards, or core and additional indicators in the case of ITU recommendations. Adapting the systems to taxonomy required a detailed examination of each framework. In this analysis, the original indicators from each system were considered: for example, in the case of ISB, only those indicators that did not appear in any of the other selected systems were collected. For the UNG framework, highly generic indicators that could not be associated with city analysis, even indirectly, were excluded. Altogether, the taxonomy resulted in an initial raw list of 1215 indicators. Although this may seem like a high figure, it is the result of compiling indicators from 14 different systems, some of which complement each other, resulting in an average of approximately 87 indicators per system.

Table 4. Number of indicators and percentage by subdimension and dimension after initial collection and application of filter I (redundancy).

Dimension	Subdimension	Initial (Raw List)			After Filter I			Reduction (Initial to Filter I)
ECO	Business innovation	30	2.5%	100 8.2%	22	2.7%	62 7.7%	8	26.7%	38 38.0%
	Entrepreneurship	21	1.7%		16	2.0%		5	23.8%
	Productivity	22	1.8%		11	1.4%		11	50.0%
	Local–global interconnectivity	27	2.2%		13	1.6%		14	51.9%
HIC	Academic and digital training	87	7.2%	154 12.7%	49	6.1%	104 12.9%	38	43.7%	50 32.5%
	Creativity	8	0.7%		6	0.7%		2	25.0%
	Management of urban life	26	2.1%		22	2.7%		4	15.4%
	Work flexibility	33	2.7%		27	3.3%		6	18.2%
GOV	Transparency	34	2.8%	130 10.7%	26	3.2%	103 12.7%	8	23.5%	27 20.8%
	E-government and online services	21	1.7%		16	2.0%		5	23.8%
	Participation in decision-making	28	2.3%		19	2.3%		9	32.1%
	Innovation in municipal management	47	3.9%		42	5.2%		5	10.6%
INF	Public transport	34	2.8%	196 16.1%	20	2.5%	129 15.9%	14	41.2%	67 34.2%
	ICT infrastructures	66	5.4%		31	3.8%		35	53.0%
	Infrastructures, traffic and logistics	56	4.6%		43	5.3%		13	23.2%
	Sustainable mobility	40	3.3%		35	4.3%		5	12.5%
ENV	Energy efficiency	56	4.6%	319 26.3%	32	4.0%	197 24.4%	24	42.9%	122 38.2%
	Resource and waste management	103	8.5%		59	7.3%		44	42.7%
	Environmental protection	127	10.5%		86	10.6%		41	32.3%
	Renewable energy–social awareness	33	2.7%		20	2.5%		13	39.4%
SOW	Public, social and security services	89	7.3%	316 26.0%	66	8.2%	214 26.5%	23	25.8%	102 32.3%
	Tourism, culture and leisure	49	4.0%		39	4.8%		10	20.4%
	Social cohesion and inclusion	104	8.6%		66	8.2%		38	36.5%
	Health and welfare	74	6.1%		43	5.3%		31	41.9%
TOTAL		1.215			809			406		33.4%

shows the results of the number of indicators by dimension and subdimension of the first raw list. The complete compilation of indicators classified according to taxonomy is provided in Supplementary S1.

4.2. Indicator Selection Process

The compilation of indicators from the selected systems obtained is adapted to the defined taxonomy, but it is a raw collection, so the defined filters are applied to obtain a refined selection.

4.2.1. Filter I: Redundancy

The first redundancy filter is applied to this initial raw list. Table 4 shows the results before and after application, gathering the total indicators by subdimension and dimension and the percentage weight of the indicators for each subdimension and dimension, as well as the reduction in the number of indicators that occur in the process. Although the aim of this study is not to analyze indicator systems themselves, a task already undertaken with different scopes in previous works [22,31,54], it was considered relevant to examine the nature of the selected indicators in relation to the different stages of the process.

The distribution of raw indicators shows a strong emphasis on Environment and Energy and Social Services and Safety dimensions both directly linked to sustainability [33]. Together, these two dimensions comprise more than half of all indicators, exceeding by over 60% what would be expected in a balanced distribution. The need for greater consideration of social and environmental aspects in smart cities, as highlighted in scientific literature [54], therefore seems to be clearly reflected in the most influential and established collections of indicators [61]. By contrast, Economy, the other core pillar of sustainability [62], is the least represented dimension, with fewer than half the expected indicators. Governance and Human Capital dimensions also receive comparatively limited attention, despite their recognized importance in smart city transformation [63,64] and their ties to innovation and urban development [65].

Applying Filter I to remove duplicates (i.e., indicators measuring the same variable) reduced the average total by about 33%, showing that roughly one-third of indicators across systems are overlapping. This result confirms the existence of a fairly significant general consensus on smart city indicators, but also the proliferation of overlapping metrics between indicator systems of different natures. The distribution of indicators by dimension changed very little after removing redundancies, although it is noteworthy that the dimension with the fewest indicators, i.e., Economy, was also among the most affected by redundancies, further accentuating its comparatively low representation. By contrast, the other underrepresented dimension, Governance, showed a markedly lower level of overlap, suggesting greater variability in the metrics used to assess it.

By thematic area, the highest redundancy was found in ICT infrastructure indicators, where more than half of the collected metrics overlap. This reflects that frameworks adopt very similar approaches to measuring connectivity and digital infrastructure development. A similar pattern appears in two aspects of the Economy dimension, productivity and local–global interconnectivity, where indicators often replicate the same macro-level economic metrics (e.g., GDP contribution, business density, export capacity) even though they are not easily adaptable to project-scale evaluation. The Environment dimension overall shows the largest value of duplicate indicators, with particularly high redundancy in those related to energy efficiency and waste and resource management. This high repetition is explained by the maturity and standardization of environmental measurement practices, which have long converged on similar metrics.

The three subdimensions with the most indicators after redundancy filtering were Environmental Protection, Social Services, and Social Cohesion and Inclusion, which together represent about 25% of the total. This highlights once again the strong focus on sustainability-related metrics, both environmentally and socially. Other areas with notable weight include Resource and Waste Management and Academic Training. By contrast, Creativity, Productivity, Local and Global Interconnectivity, and Entrepreneurship showed far fewer indicators than would be expected in a balanced distribution. Technology-related aspects also did not stand out significantly, suggesting that, across the selected systems, the smart and sustainable city is increasingly conceived from a socially oriented rather than technology-driven perspective, a trend already noted in the academic context [32]. Recent research focused on the analysis and development of indicator sets has also reported very high concentrations in the social and environmental dimensions exceeding 50%, closely matching the proportions found in this study [34,42]. Other studies, using different taxonomies, similarly highlight the predominance of so-called soft aspects, those not centered on physical infrastructure and technology [66], across most indicator collections analyzed [22,43].

While the overrepresentation of environmental and social dimensions underscores the strong orientation of current frameworks toward sustainability, the relative neglect of Economy and Governance raises critical concerns. The economic dimension is one of the three components of sustainability [42], and in the evaluation and management of individual projects, especially in scenarios of resource scarcity, it is even more important [12]. The weakness in how economic aspects are addressed in indicator frameworks is also highlighted in studies focused on these analyses [13]. Economic indicators are essential to capture the financial sustainability and competitiveness of projects, while governance indicators are key to ensuring accountability, transparency, and long-term legitimacy of urban initiatives [18,67]. Their underrepresentation implies the risk of overlooking factors that are crucial to the viability of projects [13] and public confidence, especially in small and medium-sized cities with limited resources.

The subdimension-level results of the filter I application are provided in Supplementary S1.

4.2.2. Filter II: Relevance for Project Evaluation

The application of Filter II represents the first step toward a specific focus on project-level evaluation of smart cities, as opposed to general city-level assessment. The results indicate that the relative weight of indicators across dimensions and subdimensions (

Table 5. Number of indicators and percentage after applying filter II by dimensions and sub-dimensions, and main contributions of the indicator systems to the final framework.

Dimension	Subdimension	After Filter II		ISO	ITU	CKE	UNG	CSCP
ECO	Business innovation	16	2.5%	8.1%	24.3%	24.3%	16.2%	5.4%
	Entrepreneurship	8	1.2%
	Productivity	7	1.1%
	Local–global interconnectivity	6	0.9%
HIC	Academic and digital training	28	4.3%	22.1%	11.8%	25.0%	10.3%	16.2%
	Creativity	5	0.8%
	Management of urban life	16	2.5%
	Work flexibility	19	3.0%
GOV	Transparency	23	3.6%	13.6%	12.5%	42.0%	11.4%	2.3%
	E-government and online services	13	2.0%
	Participation in decision-making	17	2.6%
	Innovation in municipal management	35	5.4%
INF	Public transport	17	2.6%	40.8%	17.5%	12.6%	0.0%	2.9%
	ICT infrastructures	17	2.6%
	Infrastructures, traffic and logistics	39	6.1%
	Sustainable mobility	30	4.7%
ENV	Energy efficiency	27	4.2%	39.6%	15.4%	23.7%	13.6%	0.0%
	Resource and waste management	54	8.4%
	Environmental protection	70	10.9%
	Renewable energy–social awareness	18	2.8%
SOW	Public, social and security services	52	8.1%	40.2%	11.7%	12.8%	14.5%	1.1%
	Tourism, culture and leisure	33	5.1%
	Social cohesion and inclusion	55	8.5%
	Health and welfare	39	6.1%
TOTAL		644	100%

) does not differ significantly from the distribution observed after the previous filter, in terms of its influence on this new focus. Nevertheless, the application of this relevance filter establishes the final contribution of each indicator system to the project evaluation framework developed in this study. Table 5 presents the percentage of influence by dimension of the five indicator systems with the highest values.

The ISO standards collection, including ISO 37120, ISO 37122, and ISO 37123, is the indicator framework with the greatest overall influence across all dimensions, particularly in Infrastructure, Environment, and Society, where nearly 40% of the selected indicators belong to this set of standards.

However, its influence is considerably lower in the Economy dimension, where the ITU frameworks and the CKE evaluation model together account for 50% of the selected indicators. The latter also shows consistently high values across all dimensions, which is unsurprising given that it is a highly comprehensive system and the only one that includes a specific collection of indicators for project evaluation. In the Governance dimension, it represents more than 40% of the selected indicators.

The ITU collections (1601, 1602, and 1603) also display relatively high average values, being the only system that maintains an influence greater than 10% across all dimensions. This makes it a comprehensive framework, not limited to ICT-related aspects within the infrastructure dimension, as might otherwise be expected from its origin [68].

The UNG indicators likewise show notable influence across all dimensions except Infrastructure, with particularly strong representation in Economy, Society, and Environment, the three thematic areas most directly linked to sustainability [62]. Finally, the CSCP collection has a distinctive impact on the Human Capital dimension, where its comprehensive set of indicators related to education and training provides substantial coverage.

In any case, the indicators selected after the application of Filter II are drawn from diverse and complementary sources. Rather than reflecting the direct transfer of a single normative framework, they represent the intended synthesis of different perspectives on the smart and sustainable city, incorporating varied approaches (Figure 3).

4.3. Defining Project Evaluation Areas: Incorporating Project Actions

The collection of indicators developed in this study evaluates projects, which are in turn broken down into the project actions that comprise them [26]. For this reason, a comprehensive list of these actions is entered as an input, in this case, the catalog developed by the ASCIMER project [26]. The first step in checking the completeness of this list of project actions and their adaptation to the framework taxonomy will be to classify the project actions according to this same taxonomy. A total of 133 actions were considered, which, when organized by dimension and subdimension, covered all categories of taxonomy. At the dimension level, the number of actions ranged from a maximum of 32 to a minimum of 14, while at the subdimension level, the range extended from 16 to 3. It can therefore be concluded that the list of project actions used is adapted to the expected levels of heterogeneity and subdimension coverage.

The reclassification of project actions according to the taxonomy allows us to perform two cross-analyses to detect gaps in the collection of indicators on the one hand and define areas for evaluation on the other.

4.3.1. Gaps in the Set of Implementation Performance Indicators

Cross-referencing the reclassified project actions with the results of the indicator selection after Filter II yielded, as a first outcome, the detection of indicator gaps (

Table 6. Number of gaps detected after the cross-referencing with project actions by subdimension and dimension, including the percentage of total gaps.

Dimension	Subdimension	Gaps Detected
ECO	Business innovation	0	10 22.2%
	Entrepreneurship	5
	Productivity	1
	Local–global interconnectivity	4
HIC	Academic and digital training	3	7 15.6%
	Creativity	1
	Management of urban life	2
	Work flexibility	1
GOV	Transparency	7	11 24.4%
	E-government and online services	1
	Participation in decision-making	0
	Innovation in municipal management	3
INF	Public transport	2	12 26.7%
	ICT infrastructures	0
	Infrastructures, traffic and logistics	10
	Sustainable mobility	0
ENV	Energy efficiency	1	1 2.2%
	Resource and waste management	0
	Environmental protection	0
	Renewable energy–social awareness	0
SOW	Public, social and security services	2	4 8.9%
	Tourism, culture and leisure	2
	Social cohesion and inclusion	0
	Health and welfare	0
TOTAL		45

). These gaps represent the variables identified as necessary to measure project actions within each subdimension.

The dimensions with the largest number of detected gaps coincide with those that had the lowest number of indicators after the application of the previous filters. Together, Economy, Governance, and Human Capital account for almost two-thirds of the new indicators, with the particular case of the infrastructure dimension, which required the addition of 10 indicators concentrated in a single subdimension (Urban Infrastructure, Traffic, and Logistics). Indeed, within the Economy dimension, the thematic areas of entrepreneurship and local–global interconnectivity, and within the Governance dimension, those of transparency and innovation in municipal management, account for more than half of the remaining indicator gaps identified.

The new indicators developed to complete the project outcome collection represent 6.5% of the full set. Considering that most of the systems analyzed were not originally designed for project evaluation, this is a remarkably low figure, which demonstrates the level of comprehensiveness already provided by the selected evaluation frameworks [13]. Moreover, developed indicators are based on highly specific variables within each identified evaluation area.

Cross-analysis between the selected indicators on the one hand and the project actions on the other allows indicators to be assigned to each project action, thereby identifying gaps for the evaluation of the latter. This analysis is carried out for each subdimension individually. The process for defining performance indicators after implementation consists of identifying the specific variable required for this gap, and based on it, developing the simplest possible indicator that aligns with it. The complete list is provided in Supplementary S2. To illustrate, two examples of indicators belonging to Entrepreneurship and Transparency and citizen communication channels subdimensions:

• Percentage of entrepreneurs who manage to access public aid for setting up their business within the first three months of applying.
• Percentage of positions in municipal administration, both technical and political, in which the incumbent lacks specific prior training for the position held.

4.3.2. Identification of Assessment Areas for Project Impact Indicators

A further outcome of incorporating project actions was the determination of the evaluation areas that serve as the basis for the subsequent development of impact indicators. In this case, the cross-analysis between project actions and the indicators selected in each of the subdimensions consists of defining homogeneous sets of indicators and project actions that define an evaluation area on which to develop an expected impact indicator. This cross-referencing process identifies 73 areas for evaluation (

Table 7. Project evaluation areas by subdimension, ECO.

Dim.	Subdim	Assessment Areas for Project Impact Indicators	Type 1	Type 2	Metric
ECO	BLI	Business Innovation	Indirect	Multi-aspect	Treatment–Impact
		Labor Market Innovation	Indirect	Specific	Employment in innovation
	ENT	Policies, Support, and Public–Private Collaboration in Entrepreneurship	Indirect	Specific	Businesses created
		Physical Infrastructure for Entrepreneurship	Indirect	Specific	Space area
		Bureaucratic Procedures for Entrepreneurship	Direct	Multi-aspect	Situation improvement
	PROD	Productivity	Indirect	Specific	GDP per worker (sectors)
	LGI	Online Business Presence	Indirect	Specific	Number of businesses
		Business Network Development	Indirect	Specific	Number of businesses
		Export and Globalization	Indirect	Multi-aspect	Treatment–Impact

,

Table 8. Project evaluation areas by subdimension, HIC.

Dim.	Subdim	Assessment Areas for Project Impact Indicators	Type 1	Type 2	Metric
HIC	ACA	Digital Education	Indirect	Multi-aspect	Treatment–Impact
		Academic Education	Indirect	Multi-aspect	Treatment–Impact
	CREA	Employment in Creative Industries	Indirect	Specific	Employment in creative industries
		Fab Labs and Living Labs	Indirect	Specific	Space area
	URL	Urban Activation Spaces	Indirect	Specific	Treatment–Impact
		Online Platforms for Public Activities and Facilities	Direct	Specific	Number of available facilities
		Citizen Participation in Public Life	Indirect	Specific	Participation
	WOR	Employment	Indirect	Specific	Employment generated
		Employment Conditions and Work–Life Balance	Indirect	Multi-aspect	Treatment–Impact

,

Table 9. Project evaluation areas by subdimension, GOV.

Dim.	Subdim	Assessment Areas for Project Impact Indicators	Type 1	Type 2	Metric
GOV	CCC	Open Data and Sensor Network Information	Direct	Multi-aspect	Situation improvement
		Transparency and Public Procurement	Direct	Multi-aspect	Situation improvement
		Data Privacy and Protection	Direct	Specific	Level of protection
		Anti-Corruption Measures	Direct	Multi-aspect	Situation improvement
		Citizen Communication Channels for Municipal Services	Direct	Multi-aspect	Situation improvement
	EGOR	E-Government Services Platform	Direct	Multi-aspect	Situation improvement
		Municipal Service Apps	Direct	Multi-aspect	Situation improvement
	PART	Democratic and Civic Participation	Direct	Specific	Participation level
		Engagement of Urban Stakeholders in Initiatives	Direct	Specific	Participation level
	MUM	Efficiency in Citizen-Oriented Municipal Management (External)	Direct	Multi-aspect	Situation improvement
		Organizational Innovation for Internal Municipal Efficiency	Direct	Multi-aspect	Situation improvement
		Project Planning and Management	Direct	Specific	Implementation level
		Urban Strategy and Planning	Direct	Specific	Implementation level

,

Table 10. Project evaluation areas by subdimension, INF.

Dim	Subdim	Assessment Areas for Project Impact Indicators	Type 1	Type 2	Metric
INF	PUBTR	Quality and Accessibility of Public Transport	Direct	Multi-aspect	Situation improvement
		Integration of Public Transport and Multimodal Systems	Direct	Multi-aspect	Situation improvement
	ICT	Data Collection and Analysis Systems	Direct	Multi-aspect	Situation improvement
		ICT Infrastructure Quality	Indirect	Specific	Quality and coverage
	INFRA	Urban Logistics	Indirect	Specific	Logistic flows
		Traffic Management Solutions	Indirect	Multi-aspect	Treatment–Impact
		Road and Drainage Infrastructure	Direct	Multi-aspect	Situation improvement
		Building Development and Land Use	Indirect	Multi-aspect	Treatment–Impact
	MOB	Non-Motorized Mobility (Cycling and Pedestrianization)	Direct	Multi-aspect	Situation improvement
		Shared and Rental Vehicle Solutions	Indirect	Specific	Vehicle usage
		Motorized Mobility with Clean Energy	Indirect	Specific	Vehicle usage
		Reduction in Private Motor Vehicle Traffic	Indirect	Specific	Vehicle usage

,

Table 11. Project evaluation areas by subdimension, ENV.

Dim	Subdim	Assessment Areas for Project Impact Indicators	Type 1	Type 2	Metric
ENV	ENE	Energy Efficiency in Public Devices	Direct	Specific	Energy consumption
		Energy Efficiency in Public and Private Buildings and Facilities	Indirect	Specific	Energy consumption
		Public Lighting	Direct	Specific	Energy consumption
		Electricity Distribution Networks	Indirect	Specific	Treatment–Impact
		Reduction in Energy Demand	Direct	Specific	Energy consumption
	RWM	Waste Management and Recycling	Direct	Multi-aspect	Situation improvement
		Water Treatment and Consumption	Indirect	Multi-aspect	Treatment–Impact
		Use of Recycled, Reused, and Renewable Materials	Direct	Multi-aspect	Specific treatment
		Food Production and Urban Agriculture	Indirect	Multi-aspect	Treatment–Impact
	ENPR	Green and Blue Urban Zones	Direct	Specific	Area of zones
		Natural and Biodiversity Protection	Direct	Multi-aspect	Treatment–Impact
		Air Quality	Indirect	Specific	Air quality
		Disaster Resilience and Risk Management	Direct	Multi-aspect	Situation improvement
		Noise Pollution	Indirect	Specific	Noise pollution
		Sustainable Procurement Criteria for Goods and Services	Direct	Specific	GPP implementation
	RENE	Renewable Energy	Indirect	Specific	Use of renewable energy
		Consumption Habits and Public Awareness	Indirect	Multi-aspect	Treatment–Impact

and

Table 12. Project evaluation areas by subdimension, SOW.

Dim	Subdim	Assessment Areas for Project Impact Indicators	Type 1	Type 2	Metric
SOW	PUSER	Quality of Public and Social Services	Direct	Multi-aspect	Situation improvement
		Security and Emergency Services	Direct	Multi-aspect	Situation improvement
		Public Spaces and Facilities	Direct	Multi-aspect	Situation improvement
		Cybersecurity	Indirect	Multi-aspect	Situation improvement
	TOCUL	Tourism	Indirect	Specific	Number of stays
		Cultural and Leisure Activities	Indirect	Specific	Number of participants
		Cultural Heritage Preservation	Direct	Multi-aspect	Situation improvement
	SOC	Social Cohesion and Barrier Reduction	Indirect	Multi-aspect	Treatment–Impact
		Accessibility and Adaptability for People with Disabilities and the Elderly	Indirect	Multi-aspect	Treatment–Impact
		Social Awareness and Volunteering	Indirect	Specific	Participation level
		Poverty Alleviation	Indirect	Multi-aspect	Treatment–Impact
	HEA	Healthy Lifestyles	Indirect	Multi-aspect	Treatment–Impact
		Healthcare and Health Services	Direct	Multi-aspect	Situation improvement

).

With the detailed definition of the evaluation areas, the basis for defining the expected impact indicators of projects is established.

4.4. Defining Anticipated Impact Indicators for Smart City Project Evaluation

In addition to the guidelines established in Section 2.2 and Section 3.3 for indicator development, it was necessary to conduct a preliminary analysis of the typology of evaluation areas from which the indicators would be defined. This analysis revealed the first typology of indicators to be developed:

• Direct indicators: These derive from internal actions of the public administration, and their results are the direct consequence of those actions. No external actors are involved, and no further response is induced as the outcome is produced internally.
• Indirect indicators: These are based on actions that seek an external response, inducing reactions from other actors rather than originating solely within the public administration.

Within each of these categories, a second typology was applied according to the scope of the indicator to be developed, which depends on the breadth of the evaluation area:

• Specific indicators: These are linked to a precise aspect being measured. They can be associated with a defined magnitude or variable.
• Multi-aspect indicators: These assess several potential aspects on which an initiative may have an impact, or cases where no clear and specific magnitude exists on which to base measurement.

In sum, four categories of indicators emerge as combinations of these two typologies, which in turn shape the final morphology of the indicators (

Table 13. Categories of indicators to be developed.

	Specific	Multi-Aspect
Direct	Expected impact on a specific magnitude or aspect, internally in areas controlled by the public administration.	Expected overall situation improvement, as there is no defined magnitude, a set of aspects must be assessed. The evaluation is simplified using the concept of expected improvement sought in that set.
Indirect	Expected impact induced on a specific magnitude externally in areas not controlled by public administration.	Treatment–impact, i.e., how it is addressed in the initiative and its expected impact. As there is no single magnitude or aspect on which to assess the impact, a relationship is established with the treatment that the project gives to the set of aspects.

).

This preliminary classification guides the development and definition of the indicators toward a practical, user-oriented conception, aligned with the framework’s philosophy, since the indicator framework is designed primarily for use by public administration decision-makers and urban planners. Table 7, Table 8, Table 9, Table 10, Table 11 and Table 12 present, for each developed indicator, its classification according to Typology 1 and Typology 2, as well as the corresponding metric based on its assigned category. This analysis, together with the guidelines for drafting indicators, also facilitates the final definition of impact indicators by focusing on the characteristics of the indicator to be developed.

Unlike the post-performance indicators, the anticipated impact indicators were designed using Likert-type scales, as these allow for the structured and comparable anticipation of a project’s expected effects in areas where objective data are not yet available, given that projects are still in the planning stage. This approach facilitates the inclusion of expert judgment input in the estimation of impacts while reducing analytical complexity into a clear and manageable format. Moreover, the use of Likert scales ensures the integration of qualitative aspects, enhancing comparability across projects and supporting decision-making in early planning phases [69].

Accordingly, each Likert-type indicator was individually developed for the identified evaluation areas. Five levels were defined to reflect the degree of impact of the initiative under assessment.

For each indicator, an individual profile sheet was developed, including its description and the related project performance after implementation of the indicators compiled. These profile sheets were organized by dimension, subdimension, and evaluation area, thereby constituting the smart city project evaluation framework. As an example,

Table 14. Transparency and public procurement indicator sheet.

Dimension:	Governance
Subdimension:	Transparency and citizen communication channels
Forecast performance indicator:	Transparency and public procurement.	TRANS2
Description:	Expected impact of the initiative on improving transparency and management of public procurement. The impact of the initiative is assessed in any of the following areas:
-	Transparency and publication of municipal government data.
-	Measures to improve transparency and management of public procurement.
-	Improvements in the management and fulfillment of requests for information from citizens.
Assessment:	Likert scale on improvements expected from the initiative in one or more aspects of the description:
	1-No impact: No improvement in any aspect of transparency, or the initiative itself does not meet transparency requirements in its contracting.
	2-Low impact: The initiative contemplates compliance with transparency and contract management requirements in its process.
	3-Medium impact: The initiative includes transparency and contract management measures in its own process above current levels.
	4-High impact: The initiative includes aspects of improvement in transparency or contract management, not only in its own process but also in other initiatives and processes.
	5-Very high impact: The initiative includes specific measures in transparency or contract management with a general improvement expected over current levels.
Related performance indicators:		Reference
	The extent to which government information is published	CKE
	Índice de transparencia y gobierno abierto.	CPS
	Use of social media by the public sector to share information about regulations and to get feedback.	ITU
	Percentage of contracts completed within the established deadline compared to total contracts.	NEW
	Proportion of contracts awarded through public tender compared to other methods, such as direct award.	NEW
	Proportion of contracts published on transparency portals and/or online platforms compared to the total number of contracts awarded.	NEW

presents the formulation of the indicator for Transparency and Public Procurement, a direct and multi-aspect indicator. The complete set of 73 indicators, with their corresponding profile sheets classified according to taxonomy, is provided in Supplementary S2.

4.5. Exploratory Pre-Validation Through Case Study Application

Once the framework has been established, its use to evaluate a project follows a simple sequence. First, the project is broken down into its constituent actions. Each action is then linked to the relevant subdimensions of the smart city model and, within them, to the corresponding indicators. These indicators are assessed to obtain impact values at the indicator level, which are then aggregated by subdimension, by dimension, and finally into an overall project score. This structure enables a holistic and comparable evaluation of a project’s expected impacts across all urban dimensions. It supports decision-making by allowing planners to compare investment alternatives individually and, when applied to a portfolio of projects within a plan or strategy, to prioritize and balance initiatives according to their multidimensional contributions.

For an initial empirical pre-validation of the framework, it was applied to a set of projects and complemented with interviews on its utility and usability with representatives from the technical department and policymakers in charge of the Smart Cities area of the case study city. The application covered a sample of 14 projects from different domains, ensuring that the prevailing dimension of each project’s main objective represented the full spectrum of dimensions included in the model. For each project, the actions involved were identified, and the evaluation was carried out using the 73 indicators of the framework (results included in Supplementary S3).

A first analysis of the results confirms the framework’s validity for multidimensional project evaluation and for capturing its holistic nature. Figure 4 shows a relief chart with impact values obtained (Z axis), by project and indicator (X and Y axes), where the framework demonstrates its ability to provide a comprehensive assessment across the various aspects of the smart and sustainable city. Beyond their initial objectives, projects were evaluated by the framework in several areas, with the average number of assessed dimensions exceeding five out of a possible six.

Statistical analysis of the use of the 73 indicators shows that only three were not applied (highly specific ones, such as those related to public lighting, which were not included in the selected project set). By contrast, more than half of the indicators were used in the evaluation of at least 20% of the projects (Figure 5). On average, each project was assessed with nearly 25% of the total indicators in the framework, which represents extensive use of the framework in each assessment exercise. This demonstrates that, regardless of their primary actions, the framework enables a comprehensive evaluation of all project aspects, capturing impacts not only of their main objectives but also in a more integrated manner.

While this empirical pre-validation provides an initial demonstration of the framework’s applicability and usefulness, it is limited to a single city case study. Nevertheless, the selected projects cover a broad spectrum of smart city initiatives, and the framework itself is grounded in indicator systems of diverse natures, themes, and geographical scopes, which broadens its potential applicability beyond a single context. Applying it in wider urban settings would further strengthen its robustness and help identify and address potential shortcomings.

Regarding utility and usability, the interviews confirmed that the framework is perceived as highly straightforward to apply, particularly thanks to the advantage of linking evaluation areas with project actions, which facilitates a quick and accurate selection of relevant indicators. In the case study, ISO 37120 was used previously as the benchmark set of indicators, and interviewees highlighted that, compared to this standard, the framework offers a broader range of possibilities and alternatives for indicator selection without adding complexity to the evaluation process. The number of post-implementation performance indicators presented as options within each evaluation area also ensures adaptability to a wide range of situations, including local contexts, despite being based on international standards. At this point, it is worth noting another limitation of the framework: the potential subjectivity and inter-evaluator bias inherent to the use of Likert-type scales when estimating project impacts. Such assessments require rigor and consistency to minimize bias and should, whenever possible, be complemented by cross-checking data sources or incorporating additional perspectives. These challenges are common in evaluations of this nature, but the framework helps to mitigate them by including post-implementation performance indicators, which allow initial impact estimations to be contrasted with real outcomes and provide feedback for refining future assessments.

When comparing the number of evaluation areas applied in a hypothetical assessment of the proposed project set using the ISO collection, namely ISO 37120, 37122, and 37123, the total would reach 46 areas, compared to the 70 covered by the proposed framework. Furthermore, in terms of the availability of post-implementation performance indicators for the project analyzed, the framework offers more than four times the possibilities provided by the ISO standards. Naturally, the framework encompasses not only the ISO collection but also 11 additional indicator systems: while the developed framework includes 697 performance indicators, the compiled ISO standards contain 273. Although the ISO collection is not specifically designed for project-level evaluation, and in comparison with project evaluation, it is at a clear disadvantage, these figures nonetheless illustrate the enhanced assessment potential of the proposed framework. Another comparison with CKE, an indicator system explicitly focused on project evaluation, further underscores this point: CKE covers 51 evaluation areas, whereas the developed framework covers 70, representing a 37.2% increase in evaluation scope (Figure 6).

In summary, the development framework demonstrates its validity through its application to the project set in the case study, as well as its strong evaluation capacity in the domain for which it was designed when compared with other indicator systems. Moreover, its structure, aligned with the assessment of project actions across the dimensions and subdimensions of the smart city, ensures an agile and intuitive evaluation process, enabling the assessment of multiple aspects of projects beyond their primary objectives or main actions.

5. Conclusions

This study explores the development of an evaluation framework for smart and sustainable city projects by compiling indicators organized and classified into a taxonomy of dimensions and subdimensions that reflects the holistic nature of smart city projects in detail. The framework transfers the logic of the main existing indicator systems at the city level to the scale of urban projects. To achieve this objective, a list of 66 evaluation systems was initially compiled, and 14 indicator frameworks were selected based on their relevance and recognition in the scientific literature. The aim was to ensure a representative sample that included systems of various types, geographical areas, and perspectives, such as integrative, corporate, sustainability, technological, and resilience systems. This basis enabled the development of a comprehensive and heterogeneous project evaluation framework.

The initial raw compilation of more than 1200 indicators, processed through the proposed methodology, allowed for an initial analysis of the number of indicators across the selected systems. This analysis underscores critical issues already signaled in the literature [23,32,33] while also offering new empirical evidence. The finding that nearly one-third of indicators are redundant confirms the proliferation of overlapping metrics, but also the existence of a consensus core of smart city indicators. At the same time, the strong bias toward environmental and social dimensions contrasts with the relative neglect of economic and governance aspects. It is thus demonstrated that the needs highlighted in the scientific literature, such as placing greater emphasis on social and environmental aspects [54], viewing technology as a catalyst rather than an end in itself [42,58], and positioning citizens and quality of life at the core of the smart city [7], are indeed reflected in current indicator systems [22,43]. However, these systems still show a relative deficit in the number of indicators addressing the other pillar of sustainability, i.e., the economic dimension [13], as well as in areas fundamental to the development of the smart city model, such as governance [64] and human capital [65,66].

By systematically documenting these imbalances, this study provides a more nuanced picture of the state of indicator systems, while also justifying the development of additional project-level indicators to fill the gaps identified. In doing so, it contributes to ongoing debates on how to move from abstract visions and rankings toward evaluation tools that are actionable, balanced, and responsive to the needs of policymakers [13,19], particularly in small and medium-sized cities [20,21], supporting informed decision-making and bridging the gap between smart city strategies and their effective implementation at the project level [35].

The definition and formulation of the indicators within the evaluation framework were directly shaped by their orientation toward public management practice and decision-making. In this way, the indicators evolve from being static metrics into dynamic tools for supporting urban governance. An initial empirical pre-validation of its application in a case study further demonstrated the framework’s practical potential. The results confirmed its capacity to provide multidimensional evaluations across diverse project types, while its ease of use and adaptability compared to established standards were highlighted. Although exploratory in scope, this application reinforces the framework’s relevance as a decision-support tool and sets the basis for future large-scale validations.

The final outcome is an integrated framework for the evaluation of the expected impacts of smart city projects, complemented by an annexed collection of metrics for assessing project performance once implementation is complete. This dual functionality strengthens the tool’s potential to close the loop between urban planning and management, enabling accountability, organizational learning, and the continuous improvement of smart city policies. Unlike traditional systems focused on rankings, international comparisons, or maturity assessments, the framework developed here is designed to inform concrete decisions about investments and specific actions, particularly in the context of smaller cities that have fewer technical and financial resources but face equally complex challenges of sustainability, economic competitiveness, and social well-being. The resulting framework strikes a balance between comprehensiveness and usability, marking a relevant step in the transition of smart cities from an abstractly evaluated concept to a tangible, results-oriented practice at the project scale.

Nevertheless, the construction of the indicator collection is based on normative sources and internationally recognized reference frameworks, which may not fully capture the local particularities of each city or region. In fact, project impact evaluation is part of a comprehensive assessment. This impact evaluation framework should therefore be complemented by project prioritization tools that incorporate local specificities and the active involvement of key urban stakeholders. Therefore, one future line of research is the integration of the proposed framework into prioritization tools tailored to specific cities. Another limitation, and potential line of inquiry, concerns the number of impact indicators: the current balance between specific and multi-aspect indicators (50–50) reflects a deliberate effort to simplify the evaluation process. However, a larger number of indicators could provide greater detail in certain areas of evaluation and reduce the subjectivity introduced by Likert-type indicators in such cases. The expansion of the initial empirical pre-validation developed in this study to a broader scope would allow these potential shortcomings to be identified and corrected. Finally, exploring the integration of the framework into dedicated software platforms for urban planning and project evaluation and leveraging the potential of artificial intelligence and big data could significantly enhance its usability and adoption. Developing such digital tools would make the framework more accessible to practitioners, facilitate consistent application across diverse urban contexts and contribute to establishing it as a useful reference for evaluating smart city projects in small- and medium-sized cities.