Proposal for a Comprehensive Tool to Measure Smart Cities under the Triple-Helix Model: Capacities Learning, Research, and Development

Abstract

This paper discusses the measurement of smart cities using efficiency indices and proposes a comprehensive tool based on the triple-helix model to assess the learning, research, and development capabilities of smart cities. Existing smart city models are divergent and lack alignment, making it difficult to compare and evaluate cities. The proposed tool aims to contribute to science, technology, and innovation policies by assessing the capabilities of participants in the regional innovation system of smart cities. The study follows a non-experimental, cross-sectional, and descriptive methodology consisting of three stages: identification of variables, definition of variables, and construction of the tool. It finds that current smart city indicators focus primarily on technological aspects, efficiency, and management processes, overlooking important factors such as citizen engagement, their capacity to adopt technologies, and their research and knowledge-generation capabilities. This study makes a significant contribution to the field of smart city measurement and evaluation by using the triple-helix model as a conceptual framework. This approach strengthens the existing knowledge about this phenomenon and lays the foundation for future research in this area.

1. Introduction

Currently, more than half of the world’s population lives in cities, and this proportion is expected to reach 60% by 2030. Cities and metropolitan areas are centers of economic growth contributing approximately 60% of global GDP. However, they also account for about 70% of global carbon emissions and consume more than 60% of resources [1]. As a result, rapid urbanization is leading to increasing numbers of people living in poor neighborhoods as well as inefficient and failing infrastructure and services (e.g., waste collection and disposal, water and sanitation, and roads and transportation), exacerbating air pollution and uncontrolled urban sprawl [2].

Therefore, the term smart city has emerged to use technology as an essential factor for cities to keep up with the pace of societal transformation, provide a solution to meet the needs of their residents, and fulfill their expectations [3]. This concept has been shown to increase the efficiency of urban centers and also improve the management of their resources through more participatory processes [4].

The concept of the smart city has been studied from the perspective of technology, people, and institutions [5,6] and has been directly related to the adoption and implementation of technologies. Information and communications technologies (ICT) play an important role in the vision and construction of a smart city. They are used to implement projects focused on urban planning and to improve the quality of life of the inhabitants; in addition, they generate data that reflect the efforts made to achieve the goal of improving the economic, social, and environmental sustainability of cities [7]. However, the role of human capital and its capabilities in the development of territories or smart cities should also be considered [8]. According to Kummitha and Crutzen [9], smart cities need to focus on people and their capabilities rather than just on the adoption and implementation of ICTs. They argue that improving of the skills and capabilities of communities will lead to the active creation and use of technologies, creating an efficient interaction between people and technology.

In addition, a city and/or territory must go through a specific process to become a smart city [10]. Such a process involves a series of conditions that must be incorporated into the regular dynamics of cities, which includes knowing the relationships between their actors, their strengths, their weaknesses, and the level of development of their capabilities [11]. As a result, cities can develop strategies, plans, and/or policies that contribute to the construction of the vision of the city they have established [12].

Therefore, this study proposes a tool to measure the learning, research, and development capabilities of smart cities under the triple-helix model to contribute to the diagnosis of the current state of these capabilities. For this purpose, this paper presents a conceptual framework that defines smart city, learning capabilities, and the triple-helix model. It then details the methodology adopted here (divided into five stages) to create the measuring tool using a set of indicators and also shares the results. The conclusions include recommendations for the application of indicators depending on the context and the particular dynamics of the city in order to measure the management and efficiency of the city based on capabilities that are not only related to technology assimilation and implementation.

2. Literature Review

According to Jonek-Kowalska and Wolniak [13], the concept of a smart city has been present in the literature since the 1980s and aligned with the development of information technologies. In studies on smart cities, two dominant approaches have been distinguished: a holistic human approach and a techno-centric approach [14]. In recent years, the latter approach has been dominant in European policy agendas [15]. From this perspective, innovation models have been proposed as conceptual forms of the current rhetoric of smart cities driven by innovation [16].

According to Paskaleva et al. [17], three phases of smart cities have been distinguished. The first is enterprise-driven, the second is local-government-driven, and the third is citizen-driven. Enterprise-driven smart cities are characterized by technological impetus from enterprises, while local government-driven smart cities are considered suitable to address current problems, and citizen-driven smart cities adopt co-creation strategies. Recently, some cities such as Amsterdam [18], Barcelona [19], Vancouver [20], and Medellín [21] have adopted models based on the local government and citizen approach.

On this basis, models such as the triple helix have emerged, which have made it possible to study the knowledge base of an urban economy about society and development as the main components of innovation systems [22]. Thus, it is argued that cities can become smart if universities and industry support government investment in the development of traditional communication infrastructure, i.e., transportation and modern infrastructure, i.e., ICT, to promote the city’s economic growth as well as sustainability and quality of life [23].

Motivated by this relationship, some research studies, such as that of Dameri et al. [24], have identified the similarities and differences in the vision of the smart city among the main actors, namely public agencies, universities, and private companies, through an extensive and in-depth literature review. One of the most important contributions to the modeling of smart city performance was proposed by Lombardi et al. [25], who offered a profound analysis of the interrelationships between the components of smart cities that connect the pillars of the triple helix, aiming to model, categorize, and begin to measure the performance of smart cities.

Other authors such as Alderete [26] have studied the ICT factors to assess the technological innovation of a city and found that the ICT development of smart cities depends both on the characteristics and capabilities of the cities themselves and macro-technological factors. Furthermore, this emphasizes that there is a lack of studies and methodologies for evaluating the indices and performance of smart cities in the literature. A useful theoretical framework for analyzing the smart city phenomenon in a broader sense is the triple-helix approach. This model brings the perspective of the knowledge economy, and by integrating key actors such as businesses, universities, and government, it is suitable for analyzing the dimensions included in smart cities and understanding the role it plays in measurement.

2.1. Triple-Helix Model

The triple-helix model emerged as a reference framework for the analysis of knowledge-based innovation systems. Such a model combines multiple and reciprocal relationships between the three main actors in the process of knowledge creation and capitalization: university, industry, and government [27,28]. Dameri, Negre, and Rosenthal-Sabroux [24] stated the following:

(...) The triple helix concept interprets the shift from a dominating industry; government dyad in the Industrial Society to a growing triadic relationship between university-industry-government in the Knowledge Society. The Triple Helix thesis is that the potential for innovation and economic development in a Knowledge Society lies in a more prominent role for the university and in the hybridisation of elements from university, industry and government to generate new institutional and social formats for the production, transfer and application of knowledge.

According to Etzkowitz and Klofsten [29], the triple-helix model consists of three basic elements. First, it assumes a more prominent role for the university in innovation. Second, it is aware of the movement towards collaborative relationships between the three parties, in which innovation policy gradually becomes a result of interaction. Finally, each party “takes the role of the other”, operating on a y-axis of its new role as well as on an x-axis of its traditional function. It turns out that an entrepreneurial university assumes the roles of industry and government and represents the central institution of an innovative region.

In this sense, those institutions that assume non-traditional roles must be considered innovative, as opposed to innovation models where deviations from traditional roles are not considered important. Especially in the triple-helix model, the academy has a relevant participation in the creation of companies and regional development. In addition, it has been shown that it gives support as a supplier and as a trainer of human talent. In this sense, the two remaining parties assume their role in the model as support for new developments based on regulations, tax incentives, and the provision of public risk capital by the government. Industry oversees training and research, probably at the same level as universities [29].

However, the triple-helix model has evolved to better understand the interaction between the agents that make it up. Today, authors refer to the quadruple-helix model, which introduces society and unifies the three initial agents. Consequently, Cohendet and Simon [30] claimed that it is not only universities, industries, or governments but communities that should provide the environments so that cities can successfully take advantage of the opportunity to manage the integration produced by these interactions.

2.2. Smart Cities

The world faces major challenges, including poverty eradication, inequality reduction, climate change mitigation, and affordable and clean energy, which have a direct impact on the survival of the planet and its people [31,32]. To address these challenges, a number of policies and strategies have been established at the global level, such as the UN Sustainable Development Goals and climate change agreements, which aim to mitigate or even eliminate these problems and thus improve the quality of life of people.

According to the United Nations [30], the world’s population grew from 1500 million in 1900 to 7347 million in 2015 and is expected to increase to 9224 million in 2050, mainly in urban areas. Cities now concentrate 55% of the world’s population, generate 70% of gross domestic product (GDP), consume more than 60% of energy, produce 70% of the solid waste, and generate 70% of greenhouse gas emissions. This demonstrates both the economies of scale of metropolitan areas and the environmental and sustainability risk to which the planet is exposed [33].

Although the concept of the smart city has provided an alternative to address the problems generated by this rapid demographic growth, it also raises issues related to its concepts. In particular, there is a clear problem of conflating smart cities with a range of terms such as cyber, digital, wired, knowledge cities, etc., when in fact these different ideas themselves have somewhat different meanings. Even though they integrate IT in their development, they are not limited exclusively to this component [34].

Other elements that should be considered in this type of city model are people and communities, which are essential parts in the construction of smart cities. Thus, this model changes the perspective that puts more emphasis on the understanding of technological and political aspects than on the full participation of civil society in the search for solutions to current problems in a given territory. Smart city projects have an impact on the quality of life of citizens and aim to foster more informed, educated, and participatory citizens. In addition, smart city initiatives allow members of the city to participate in the governance and management of the city and become active users, thus transforming them into key actors that can influence the success or failure of the effort [35].

Few authors have examined the topic of smart cities from the perspective of regional innovation systems, and even fewer have proposed indicators that integrate both concepts [25,27]. Proof of this is that the indicators and measurement tools found in this study are focused on measuring the efficiency of the technology that has been implemented or indicators that are not relevant enough to be used as a starting point formulating plans, strategies, and/or policies that promote the vision and construction of a smart city from the standpoint of the triple-helix model [36,37].

The Center for Globalization and Strategy and the Strategy Department of IESE Business School [37] created a platform called Cities in Motion, which aims to promote local change and develop valuable ideas and innovative tools to make cities more sustainable and smarter. Such a model is based on four key factors: sustainable system, innovative activities, citizen equality, and connected territory. This model defines a series of steps, such as diagnosis of the situation, formulation of a strategy, and subsequent implementation, all using the Cities in Motion Index (CIMI), which was designed to measure the future sustainability of the most important cities in the world as well as the quality of life of their inhabitants.

Lombardi et al. [25] provided an in-depth analysis of the interactions between the components of a smart city, linking the cornerstones of the triple-helix model (university, industry, and government). This analysis was extended using the analytical network process to model, classify, and measure the performance of smart cities. As a result of their study, they obtained a model that allows interactions and feedback within and between clusters, including priority scales for the elements in the system under study, thus providing a more truthful and realistic representation to support the formulation of policies.

Smart cities have been analyzed from the perspective of interconnected cities (cities in motion) and interactions among participants in regional innovation systems. However, the indicators or metrics of these two perspectives do not coincide or intertwine. Some authors [22,24,27] have presented smart cities as an evolution of innovation systems, where learning, research, and development capabilities generate new urban policies that offer opportunities for local development through the incorporation of technology.

Recently, the topic of smart cities has gained great attention in scientific research, given the fundamental purpose of improving the quality of life of citizens in urban areas. For this very reason, it is also becoming a key reference point for urban planning in large cities around the world. Thus, according to Dameri et al. [24], success of the smart city depends on the synergetic action of the key actors of the triple helix: public universities and private companies. However, these actors do not always share the same smart city vision (...) (p. 2974).

In the same way, Leydesdorff and Deakin [22] mentioned that cities can be considered as densities in networks between three relevant dynamics, where various quite important actors intervene: universities with intellectual capital, industry with wealth creation, and the government with democracy in society. The authors also mention that “these interactions generate dynamic spaces within cities where knowledge can be exploited to bootstrap the technology of regional innovation systems” (p. 53). Thus, dynamic spaces lend themselves to creating smart cities through the generation of information and communication technology (ICT) spaces, in which knowledge assumes a key position in regional innovation systems [24].

Although the ICTs play an important role in the vision and construction of smart city, human capital is also highly important in its conception. Thus, Kummitha and Crutzen [9] referred to the rationalist or pragmatic school of thought, arguing the following:

(...) smart cities would need to focus on people and their capabilities more than just concentrating around ICTs or technology. The scholars in this school imagine community-driven smart city building and argue that the enhancement of skills and capabilities of communities would result in their active creation and usage of technologies. Thus, an integrative mechanism is required which mediates human interaction with technology. (p. 46)

Therefore, science, technology, and innovation (STI) policies should be clear and structured, contribute to the strategic planning of territories, consider variables and participants, foster communication so that citizens and companies become involved in city projects, and try to improve their efficiency and sustainability. Likewise, every city is unique and has its own needs and opportunities. Therefore, plans must be designed and adapted to changes in every city’s environment, enabling the generation of networks that involve citizens, organizations, institutions, government, universities, industries, experts, and research centers (among other participants in the regional innovation and production system) in order to set common objectives for the construction of a smart city.

Based on the above, in this study, a smart city is defined as a territory capable of using and managing the capabilities of its participants in order to provide solutions to problems and/or needs that have been identified during the definition of the proposed city vision, integrating citizen involvement and the technological factor in a sustainable way.

In recent decades, international organizations have carried out a series of studies [27,28,37,38,39] that have proposed indices to represent the current situation of cities and their inhabitants. As a result, there currently are multiple urban indicators that, even though they present a city diagnosis, are not coherent when cities are compared from the standpoint of the triple-helix model, the role of its participants (university, industry, government, and civil society), or the generation of data that contribute to the evaluation of the efficiency of science, technology, and innovation (STI) policies applied in such cities.

In order to connect the dimensions of smart cities, the participants in the triple-helix model, and their capabilities, this study proposes a tool to support the process of internal city diagnosis and the formulation of STI policies aimed at building smart cities that actively integrate their citizens. Such a tool can be used to collect and present indicators that contribute to STI policymaking; however, it is up to the responsible entities to use it to support this process.

2.3. Smart City Dimensions

Smart cities present a set of dimensions or “smart components” that constitute them.

Table 1. Dimensions of smart cities.

Author	Summary	Dimension
[35]	They proposed a framework that can be used to characterize how to envision a smart city and design initiatives, which advance this vision by implementing shared services and navigating their emerging challenges. It includes eight key factors, among which technology can be considered a goal.	Management and organization Technology Governance Policy People and communities The economy Built infrastructure Natural environment
[25]	They identified six dimensions of a smart city, which connect with traditional regional and neoclassical theories of urban growth and development. In particular, the dimensions are based on theories of regional competitiveness, transport and ICT economics, natural resources, human and social capital, quality of life, and participation of citizens in the governance of cities.	Smart governance Smart economy Smart people Smart living Smart mobility Smart environment
[40]	They defined a smart city model that takes into account not only sectorial aspects but also the human, technological, and institutional factors involved in its development.	Smart economy Smart environment Smart governance Smart people Smart mobility Smart living
[4]	It suggested a smart city model based on four basic elements.	Connectivity infrastructure Connected sensors and devices Integrated operation and control centers Communication interfaces
[41]	Spain proposed a model for the promotion of smart cities in that country, which focuses on the citizen–business relationship and includes some essential fields and sub-fields.	Smart environment Smart mobility Smart governance Smart economy Smart people Smart living
[38]	They listed a set of variables or “smart components” that compose the urban ecosystem and, according to their study, reveal a consensus on the topic.	Smart economy Smart citizens Smart governance Smart mobility Smart environment Smart living

details a series of such dimensions proposed by different authors.

It can be seen above that there is no homogeneous definition of smart city. Nevertheless, the literature review revealed a series of dimensions that some authors have in common, such as smart economy, smart mobility, smart environment, smart people, smart living, and smart governance [37,38,40,42]. However, although some dimensions are shared, there is no holistic vision of the term when it comes to defining the essential components or dimensions to the construction of a smart city. Hence, this study will implement the following dimensions: smart environment, smart mobility, smart governance, smart economy, smart people, and smart living [25,40,41].

2.4. Learning, Research, and Development Capacities in Smart Cities

2.4.1. Learning Capacity

Learning capacity refers to the assets that enable companies to transform and exploit their resources to develop innovative products or processes [43] (p. 453). On the other hand, [44] indicated that absorptive capacity allows organizations to recognize the value of new information, assimilate it, and apply it for commercial purposes based on the premise that “the organization needs prior related knowledge to assimilate and utilize new knowledge...” (p. 129).

According to this, learning or absorptive capacity represents a highly relevant aspect of an organization’s ability to create new knowledge, enabling the understanding of learning dynamics derived from the processes of assimilating and exploiting internal and external knowledge within organizations to develop their innovation potential. Moreover, learning capacity depends on knowledge-management capabilities and individual learning capabilities or is a result of the organization’s investment in R&D [45,46,47,48].

2.4.2. Research Capacity

Research can be defined as a set of intellectual and experimental activities carried out systematically to increase knowledge about a specific subject [49]. Based on this, research can be classified into two types:

Basic research consists of experimental or theoretical work conducted primarily to acquire new knowledge of the fundamentals of observable phenomena and facts without any thought of specific application or use. On the other hand, applied research also includes original work undertaken to gain new knowledge, but it is primarily directed towards a specific practical objective [50] (p. 45).

Therefore, research capacity can be defined as the set of attributes or aptitudes that an individual or organization has to systematically carry out intellectual and experimental activities in order to generate knowledge on a specific topic.

2.4.3. Development Capacity

The OCDE [50] stated the following:

“Experimental development is systematic work, based on knowledge obtained from research and practical experience and the production of additional knowledge, directed towards the production of new products or processes or the improvement of existing ones” (p. 45).

Based on this, development capacity can be defined as the set of attributes or aptitudes that an individual or organization has for transforming the knowledge obtained from basic or applied research into new products, processes, or creation of improvements.

2.5. Measuring Smart Cities

The management of smart cities involves measuring and testing the performance of integrated smart city processes. Therefore, studies on the topic have focused on proposing an evaluation mechanism. For instance, Qayyum et al. [51] proposed a conceptual framework based on Six Sigma for smart city management. Studies like that of Anthopoulos et al. [52] have systematically compared modeling and comparative evaluation approaches to assess the impact of smart cities, highlighting six frequent dimensions: people, governance, economy, mobility, environment, and life.

As expressed by Zuccardi Merli and Bonollo [53], a successful smart city requires an appropriate performance-measurement system. Therefore, they proposed a new model for measuring the performance of a smart city with practical application in smart cities in Italy and Europe. The authors highlighted that the term “smart cities” is gaining more relevance, and there is confusion about the concept. Therefore, they focused on identifying the dimensions and elements, urban intelligence metrics, performance measures, and initiatives of some smart cities [54].

The IESE Business School and the University of Navarra [37] proposed a platform that contains a conceptual model based on the study of a large number of successful cases and a series of in-depth interviews with urban leaders, businessmen, scholars, and experts involved in city development. Such a model proposes a series of steps (i.e., diagnosis of the situation, formulation of a strategy, and subsequent implementation) and uses the Cities in Motion Index (CIMI) to measure the future sustainability of the most important cities in the world as well as the quality of life of their inhabitants.

On the other hand, Arizmendi et al. [36] proposed a unified metric to evaluate and compare smart cities. Based on the analysis of more than 2000 indicators, they found that this wide variety of indicators should be homogenized and applied a five-stage classification methodology to each indicator. As a result, they constructed a tool that consists of 69 indicators classified into six dimensions (i.e., smart environment, smart people, smart mobility, smart governance, smart economy, and smart living) to measure the condition of smart cities without relying on proprietary rankings.

Escobar and Perez Hernandez [38] developed a comprehensive tool for evaluating and diagnosing cities, with the objective of promoting public policies based on the six dimensions of a smart city. In order to achieve this, the researchers gathered, analyzed, differentiated, and systematized variables and indicators from five well-known indices: ISO 37120:2014 (sustainable development of communities) indicators for city services and quality of life, the Cities in Motion Index by the IESE, the smart cities wheel developed by Boyd Cohen, AENOR UNE 170001-2, and the Observatorio de Accesibilidad Universal de los Municipios de España.

The outcome of this effort was a tool comprising 69 indicators that are categorized into six smart components: smart environment, smart mobility, smart people, smart governance, smart economy, and smart living. To validate the effectiveness of the tool, it was applied to the following cities:

• Asia: Tokyo (Japan) and Singapore (Singapore);
• Europe: London (United Kingdom), Nice (France), and Madrid and Barcelona (both in Spain);
• Latin America: Medellín (Colombia) and Rio de Janeiro (Brazil);
• North America: San Francisco (U.S.) and New York (U.S.).

3. Materials and Methods

In order to design the tool proposed here, we defined a set of indicators to measure the learning, research, and development capabilities of a smart city. According to the Dirección de Seguimiento y Evaluación de Políticas Públicas [55] an indicator can be defined as an observable and verifiable quantitative expression that can be used to describe the characteristics, behaviors, or phenomena of reality by measuring a variable or the existing relationship between a group of them. Thus, they consider that indicators can facilitate the diagnosis and monitoring processes of public policies since they can quantify changes that take place in different contexts and can be used to verify the progress towards certain objectives in a period of time.

De la Vega [56] defined an indicator as a statistical summary measure that refers to a quantity or magnitude of a set of parameters or attributes. Accordingly, an indicator can be used to classify the units under analysis in terms of concepts, variables, or attributes.

In this study, metric is defined as the result of measuring the learning, research, and development capabilities of a smart city in a time series composed of periods of the same length. Hence, metrics can be used to monitor the changes generated by the implementation of a policy, strategy, and/or project designed to improve the capabilities under evaluation. That is, metrics determine the difference in values of an indicator in different periods of time, which can be used to visualize the progress toward the stablished goal.

3.1. Classification of the Indicators

Defining a set of homogeneous indicators enables a city to measure its management and advances compared to other cities. Hence, in order to select the indicators to be included in the proposed tool, a systematic process was conducted. The latter was supported by a series of categories and characteristics to identify, classify, and organize the indicators according to the learning, research, and development capabilities.

Table 2. Method to select the indicators.

Stage	Description
Stage 1. Identifying parameters	The literature review conducted here to create the theoretical framework identified a series of authors that propose tools for measuring smart cities. Additionally, other sources that presented indicators were also included in order to have multiple references from which to select the indicators.
Stage 2. Defining parameters and selecting indicators	The indicators proposed by the authors in several sources were selected using parameters established according to definitions of learning, research, and development capabilities.
Stage 3. Classifying indicators	The indicators were classified by the dimensions of smart city and according to the stablished parameters.
Stage 4. Adjusting indicators	Repeated indicators were discarded and the rest were adjusted according to the parameters to measure learning, research, and development capabilities.
Stage 5. Categorizing indicators	The indicators were grouped by dimensions and capabilities.

presents the five-stage process implemented here to select the indicators to be included in the tool.

3.2. Selection Process of Indicators for Measuring Learning, Research, and Development Capabilities

Stage 1. Identifying parameters: In this stage, we reviewed and analyzed the literature in the field regarding theories of smart cities, the triple-helix model, and capabilities that support the problem. As a result of this process, we identified several sources that presented indicators to measure the proposed capabilities and collect data.

Stage 2. Defining parameters and selecting indicators: According to the definitions of learning, research, and development capabilities, we established certain criteria to guide the selection of indicators to be included in the tool, which are listed in

Table 3. Parameters to guide the selection of indicators.

Learning (Acquire)	Research (Generate)	Development (Apply)
Knowledge absorption Assimilation and application of information Boosting innovation Knowledge management	Process of systematic generation of new knowledge Dissemination of new knowledge	Transformation of knowledge into solutions

.

In the previous table, a series of activities are presented to measure the capabilities of learning, research, and development in different dimensions of a smart city, according to the established definitions for each of them. Hence, these activities become the selection criteria that determine the relevance of including an indicator in the tool.

Therefore, the indicators-characterizing process is carried out by asking whether the evaluated indicator measures or can be adjusted to measure the proposed activities. If the answer is positive, it will be included in the list; otherwise, it will be refined or eliminated.

The outcome of this stage was a selection of base indicators extracted from the literature review. The publications related to the object of study were included, and those that proposed indicators that only measure the technological factor as a component of a smart city in all its dimensions were discarded. As a result, this study examined the five publications in

Table 4. Publications selected to extract base indicators.

Relevant Studies of Smart Cities	Article	Number of Indicators
[55]	“IESE Cities in Motion Index 2018”	83
[25]	“Modelling the smart city performance”	60
[36]	“Smart Cities. ¿Cómo determinar el estado de desarrollo de una ciudad inteligente?”	69
[57]	“Herramienta de diagnóstico para evaluar smart cities”	69
[58]	“Medición de las ciudades inteligentes: una propuesta desde México”	70

, which are about measuring smart cities, to select 351 indicators that constituted the basis to design the proposed tool.

Stage 3. Classifying indicators. We selected the indicators that could measure learning, research, and development capabilities based on the criteria established in the previous stage. The result of this process was table composed of 105 indicators. A total of 89 of them were proposed in articles, and 16 were found in other sources. All of them were grouped by city dimension (smart living, smart people, smart governance, smart environment, smart mobility, and smart economy) according to the authors. This stage allowed us to observe that, in general, there were similar indicators, which justifies the next stage in the selection process.

Stage 4. Adjusting indicators: In this stage, the selected indicators as well as the source of the information used in the articles were analyzed in detail. Afterward, the indicators were filtered, and the ones that were identical were removed. At the end of this process, the indicators that measured similar topics were grouped, with one selected to represent each group. Additionally, the ones that did not completely meet the proposed criterion were redefined in order to fulfill the requirements of this study. As a result, 33 indicators were included in the tool. This selection revealed that some indicators could measure more than one capability if some adjustments were made.

Stage 5. Categorizing indicators: In this stage, the selected indicators were grouped by capabilities and triple-helix agents, as shown in

Table 5. Final Selection and Categorization of Indicators.

Metrics and Indicators for Measuring Learning, Research, and Development Capacities
Dimension	Indicator Proposal	Learning	Research	Development	University	Company	State
Smart Citizenship	% of the population with access to higher education	x					x
	# of research and development centers in the city		x	x	x
	# of institutions offering free distance learning courses for citizens	x			x
	# of scientific articles published in indexed journals		x		x
	# of active researchers per year per 100,000 inhabitants		x		x
	# of patents granted per 100,000 inhabitants, according to the place of residence of the applicants			x		x
Smart Government	# of reusable open databases (excluding standards, laws, etc.) with information from the last 3 years			x			x
	Level of transparency in local public management in the last period	x					x
	% of GDP allocated to research and development		x	x			x
	% of public services, management processes, and mechanisms of electronic citizen participation			x			x
Smart Economy	% of public expenditure on scientific and technological activities as a percentage of GDP per capita	x	x	x			x
	City’s employment rate			x		x
	# of centers, institutes, technology parks, business incubators, and science and technology laboratories per urban area		x				x
	# of international academic and business events (conferences and business fairs) held in the city per year	x					x
Smart Life	% of teachers and researchers involved in international projects and exchanges	x			x
	# of programs offered per year for international mobility	x					x
	# of public and private libraries per city	x					x
	# of cultural events in the city (concerts, museums, interactive rooms, etc.)	x					x
	# of higher education programs offered	x			x
	# of institutions promoting art and culture per city	x					x
	% of the population with an intermediate level in a second language	x			x
	% of population with higher education	x			x
	% of investment in education	x					x
	% of population with Internet access in the city	x					x
Smart Environment	% of total energy derived from renewable sources, as a proportion of the city’s total energy consumption		x	x		x
	% of recycled waste per kilogram of waste produced			x		x
	# of times per year that the air quality levels established by the WHO as harmful to health were exceeded		x				x
	% of the population with reasonable access to an adequate amount of water from an improvement in their supply			x			x
Smart Mobility	# of electric charging stations for vehicles			x		x
	# of devices for monitoring and controlling mobility			x			x
	# of public transport companies with integrated mobility systems			x		x
	Average travel time		x				x
	# of public transport companies providing real-time service information to users.		x	x		x

. This stage completed the se-lection of the indicators and marked the start of the process to design the tool for measuring learning, research, and development capabilities.

The previous process allowed us to observe the behavior of the indicators according to the city dimension and the capability they assess, revealing the following:

• 45.5% of the selected indicators assess learning capacity and those were grouped within four out of the six city dimensions (citizenship, governance, economy, and smart living). This observation indicates that the indicators in the smart living dimension fully align with the measurement of learning capacity within a smart city;
• 30.3% of the indicators measure research capacity and are present in five out of the six city dimensions, excluding the smart living dimension;
• Development capacity is represented by 24.2% of the selected indicators and is grouped within five out of the six city dimensions, with the highest participation in the smart mobility dimension.

Table 6. Summary of Indicator Quantities by Capacity.

Smart City Dimension	Total Indicators *	Learning Capacity	Research Capacity	Development Capacity
Smart Citizenship	6	2	3	2
Smart Governance	4	1	1	3
Smart Economy	4	2	2	2
Smart Living	10	10	0	0
Smart Environment	4	0	2	3
Smart Mobility	5	0	2	4

The purpose of * is to take the singular or plural of the concept “Indicator”. Source: Own elaboration.

summarizes the number of indicators selected to measure each capacity. Subsequently, the findings of this process are documented.

According to the previous table, some of the selected indicators can be used to measure more than one capacity by adjusting their initial focus:

• For the smart citizenship dimension, six indicators were selected, where the indicator “Number of research and development centers in the city” was adjusted to measure two capacities (research and development);
• For the smart governance dimension, four indicators were selected, where the indicator “% of GDP allocated to research and development” was adjusted to measure two capacities (research and development);
• For the smart economy dimension, four indicators were selected, where the indicator “% of public expenditure on Scientific and Technological Activities per capita GDP” was adjusted to measure all three capacities (research and development);
• For the smart living dimension, 10 indicators were selected, and all of them measure learning capacity;
• For the smart environment dimension, four indicators were selected, where the indicator “% of total energy derived from renewable sources as a proportion of the city’s total energy consumption” was adjusted to measure all three capacities (research and development);
• For the smart mobility dimension, five indicators were selected, where the indicator “Number of public transportation companies providing real-time service information to users” was adjusted to measure all three capacities (research and development).

4. Results

In order to contribute to the management and governance of smart cities, it is crucial to configure and reshape policies and actions in order to make decisions for improvement. Therefore, previous studies have tried to propose systems or metrics to measure performance in developing countries. A notable example is the study by Alsaid [59], which made a theoretical contribution in an emerging economy case and provided performance metrics based on accounting. Triple-helix-based innovation systems highlight three relevant dynamic networks for smart cities performance: intellectual capital generated in universities, the creation and development of industries, and government support from civil society [22].

Proposed measurement mechanisms for smart cities under the triple-helix model should target each of the various stakeholder interactions for cooperation and coordination [14]. In addition, the governance of smart cities suggests a new way through the use of ICT for collaboration and networking among involved and interested parties [60]. Finally, mechanisms are recommended that shed light on the need for specialized university policies, industry/society empowerment, and financial measures to foster relationships and cooperation among stakeholders in the triple-helix innovation model [61].

This study resulted in a tool that allows interactions and feedback within and between clusters, provides a process to derive priority scales of the elements, and offers a more truthful and realistic representation of smart cities to support the formulation of policies. Such a tool can classify the performance indicators of smart cities, and it is composed of 60 indicators grouped into five dimensions (smart governance, smart economy, smart people, smart living, and smart environment). Thus, we completed the identification, collection, selection, and distribution of indicators for measuring learning, research, and development capabilities. Moreover, the collected information was used to construct a tool called MeCAID, which organizes indicators by capability, agent, and city dimension. The following section presents a characterization and description of the tool as well as the limitations that were found in this study.

Characterization of the Tool to Measure Learning, Research, and Development Capabilities

Understanding of the current and historic situation of a city or territory is essential to conduct an internal diagnosis of it. Therefore, any smart city diagnosis should start by defining the objectives that contribute to the formulation and monitoring and evaluating plans, strategies, and/or policies aimed at achieving the vision of a smart city that integrates its citizens in an active way. Additionally, comparing the current and past situation of a city or territory (considering variables, participants, needs, and opportunities present in it) can validate the effectiveness of the plans, strategies, and/or policies formulated with a specific purpose.

The tool in Figure 1 was designed based on the previous selection and analysis of smart city measurement tools that examine learning, research, and development capabilities within the triple-helix model. As a result of this process, we identified and selected indicators for the tool and proposed our definition of smart city adapted to the purpose of this study. The aforementioned tool is aimed at supporting the process of internal diagnosis of a city and/or territory, and it contributes to the evaluation and monitoring of plans, strategies, and/or policies for the construction of a smart city by measuring the learning, research, and development capabilities present in that territory.

This tool includes 39 indicators that can be classified by capability, triple-helix agent, and smart city dimension. Therefore, it has a matrix design, which enables the user to visualize the indicators and their metrics. The purpose of this tool is to diagnose the condition of the learning, research, and development capabilities of triple-helix agents in a city and/or territory. Thus, it compiles and retrieves data that contribute to decision making and/or the generation of strategies, plans, and policies that support the transformation of a city and/or territory into a smart city. In addition, said tool can diagnose a territory and/or city depending on the data that are incorporated to analyze the condition of its learning, research, and development capabilities over periods of time.

In addition, the instrument developed here is composed of three matrices containing three fixed fields (triple-helix agent, city dimension, and indicator) and two variables. (It could be more, depending on the number of periods to be evaluated.) These variables are the current period (t) and the previous period (t1); and, in case of other existing data, there could be other periods (tn). The fields mentioned above represent the data of the indicators that will be measured in different periods. Therefore, it is necessary to define the frequency of the data (monthly, biannual, or annual) in order to make a comparison with the corresponding reference period. For example, if we want to know the current state of the capabilities of the agents in each dimension of a city such as Medellín, we can take the 2020-1 period (t) and the 2019-1 period (t1) to compare them and establish a diagnosis of the current situation of the capabilities. This can be used to define the general situation of the city, which is a starting point to plan strategies and/or policies that define the vision of the smart city. Furthermore, this process can identify possible existing needs and/or gaps to achieve the objective proposed by the city in its transformation path.

In addition, the tool can be used to compare two cities. However, in order to do so, they should have similar characteristics regarding aspects such as investment in STI, re-search budget, number of inhabitants, and territory size; as a result, the measurement is fairer. This is because each city has particular dynamics that characterize it, which is reflected (in many occasions) in a higher degree of development of one of its dimensions, for instance, mobility or smart living. The tool proposed in this study to measure the learning, research, and development capabilities (see Figure 2) of a smart city under the triple-helix model is presented below.

The tool constructed here comprises six elements: three introductory components (index, contextualization, and use instructions) and three measuring components (learning capability, research capability, and development capability). The introductory components facilitate navigation, contextualize the tool, and provide instructions to use the tool. In turn, the measuring components were designed to introduce data, validate their variation, and make comments about them, which provides a diagnosis of the current condition of the capabilities.

Figure 2 shows the format of the tool’s measuring component, more specifically, the measuring matrix of the learning capability; however, the other two capabilities are presented likewise. Each matrix aims to measure the learning, research, and development capabilities of cities and/or territories that are being transformed into smart cities within the framework of the triple-helix model. To use the tool, the following recommendations should be considered:

• Historical data about the capabilities under evaluation should be available;
• The frequency of the measurement should be selected;
• The data of the selected periods should be collected;
• The data of the oldest period should be entered in field t1, and those of the most recent period should be entered in field t;
• The variation field shows the behavior of each indicator in the selected periods, which can be used to establish if it improved, decreased, or remained constant;
• According to the behavior shown in the measurement, users can add comments to improve current plans, strategies, and/or policies, thus contributing to the achievement of the proposed smart city vision;
• The tool has a simple, user-friendly interface.

This description completes the construction of tool proposed here to measure the learning, research, and development capabilities of a city and/or territory within the framework of the triple-helix model.

5. Discussion

As found in related studies, efforts are being made to fill the gaps in indicator models for smart city governance. In the model proposed by Herdiyanti [62], 29 indicators were evaluated in three different domains and seven assessment aspects in an emerging economy: public administration services, basic needs facilities, public utility facilities, internal policy, bureaucratic governance, public policy that embraces the perspective of positive societal impacts and accommodates public aspirations, and governmental regulation of open access.

In Sharifi’s study [63], 34 frameworks were examined to contribute to a better understanding and provide detailed information on the main methods and approaches used to assess the intelligence of a city. The author highlighted the most common themes: economy, people, governance, environment, mobility, life, and data. These results are consistent with the findings of this study. More specific studies, such as the one conducted by Afanasiev and Lysenkova [64], aimed to obtain quantitative characteristics of the university’s impact on the smart city, emphasizing indicators that characterize the quality of life as a crucial indicator of the development of “smart” and innovative cities.

The tool proposed here can collect and present relevant indicators for STI policy making; nevertheless, it is up to the responsible bodies to use it to support this process. Such a tool is a first contribution that should be enriched with more data to achieve better results. This shows that the dynamics of cities should be further explored in order to improve existing and propose new indicators that can contribute to the generation of policies that promote the smart city vision and its development with citizen participation. Therefore, the transformation and construction of a city needs citizens who are active members of existing smart city agents. It can be concluded that a city that involves its citizens in the construction of its vision will gain significant advantage thanks to their combined skills, which will benefit the achievement of a common goal.

Smart city indicators refer to technological aspects of a city; they are used to make it efficient, reflect management processes, and facilitate the life of its citizens. Nevertheless, these indicators do not evaluate citizens, the opportunities they have to contribute to developing solutions, their capability to appropriate and adopt technologies, or their research and knowledge-generation capability. This is because these factors are not technology-related, and hence, they are not usually studied. Consequently, future studies should include citizens and their capabilities to develop and adopt technologies in order to promote entrepreneurship and the local industry. Likewise, city-level open databases should be created so that agents can exchange information and support management processes and decision making for the construction of the proposed smart city vision.

In the Information Age, the data of many factors can be obtained in real time. This is possible thanks to big data and the Internet of Things, and it is secured using block-chains. However, in order to evaluate smart cities, many use unreliable information sources: data provided by agencies that are not part of the government, social networks, or citizen participation data. Such data are generally country- and not city-level; hence, cities should be evaluated by local research centers using different factors in a centralized, audited, and accessible way.

Within the implications of the study, both theoretical and practical aspects are highlighted. Theoretically, this study contributes to the measurement and evaluation of smart cities with the support of the triple-helix model from a conceptual framework that strengthens the existing knowledge about this phenomenon and provides a basis for future research focused on this model or the four- or five-helix models. In addition, this study serves as input for synergy among the actors of the model, contributing to the understanding of how they can work together to drive the transformation of cities into “smart” cities, generating knowledge about the dynamics and benefits of collaboration. Finally, the knowledge transfer aspect is emphasized, as the implementation of the proposed indicators could promote knowledge transfer among the actors involved, thus strengthening the learning capacity and knowledge generation in the field of smart cities.

In terms of practical implications, this study provides a holistic framework for measuring and evaluating the progress of a smart city within the context of the triple-helix model. The proposed indicators serve as input for governance decision making by relevant actors and urban planners. Similarly, the findings can be part of strategic planning for a smart city. Thus, by helping identify opportunities for improvement in the implementation of new technologies for mobility and key solutions, sectors can be strengthened by ICT.

Regarding the triple-helix model, the benefits of citizen participation are highlighted, as the proposed indicators’ implementation could contribute to improving citizens’ quality of life in aspects such as energy efficiency, sustainable mobility, accessibility, citizen participation, and innovation. This, in turn, contributes to enhancing the citizens’ overall quality of life.

Finally, this study found a series of limitations that reveal a knowledge gap and high-light the importance of further studying new ways to measure a city’s transformation process. In particular, the indicators identified here cannot be adjusted to measure the learning, research, and development capabilities in all the dimensions of smart city. Therefore, new indicators should be proposed beyond the existing literature in order to create a complete set of indicators that can measure the capabilities of triple-helix agents in all smart city dimensions. In addition, this study identified a lack of structured data sources that integrate the information of agents present in a city and/or territory.

Accordingly, the indicators are proposed in the tool, but they are not applied to a specific case because it was not possible to assign indicators to measure certain dimensions of different actors of the triple helix, which would result in a biased measurement by not including all the necessary data. In addition, there are gaps in the quality of information reported in open databases, which leaves room for further research and collective construction of the tool, along with the identified need to design indicators that can reflect the diagnosis of the proposed capacities.

6. Conclusions

The aim of this study is to propose indicators for the measurement of smart cities based on the triple-helix model using existing indicators, tools, and models. This proposal would have practical implications by facilitating the measurement, evaluation, and strategic planning of the transformation towards a smart city. Additionally, it would have theoretical implications by strengthening the knowledge and understanding of collaboration dynamics among main actors and promoting knowledge transfer in this field.

The systemic development of this research allowed us to understand that although there are different proposals for measuring smart cities, none of them focus on the perspective of learning, research, and development capacities addressed in this study. Furthermore, it was identified that the term “smart city” is generally associated with technology implementation rather than the capacities possessed by its agents to materialize the city’s vision. Similarly, it is evident that while actors are mentioned in a smart city, their measurement has mainly focused on the implementation of technological systems rather than the role each actor should play in the construction and development of the city. In other words, there are many cases where technologies are adopted and integrated, but there are few opportunities to foster their internal development by managing the potential and capacities of its inhabitants.

To conclude, although there are multiple sources of proposed indicators by entities and authors, they are not sufficient to measure all dimensions of smart cities from the perspective addressed in this research, which poses challenges for the proposed tool and highlights the need to introduce new indicators to complete and validate it. Therefore, it is relevant to consider the particular dynamics of a city when creating new indicators to measure its management process and efficiency based on its capacities rather than solely focusing on the level of technology adoption and implementation it possesses. Based on the above, the provided tool serves as an initial contribution to the measurement of smart cities, but further adjustments are needed to achieve a greater impact with the results it provides.

7. Future Works

Future lines of research are identified from this theme. The first among them is the formulation of more precise and specific indicators for particular contexts, taking into account aspects such as open innovation, collaboration, cooperation among system actors, sustainability, and social inclusion, among others [65]. In this way, indicators based on the triple-helix model can be adapted and effectively applied to different local contexts, taking into account geographical, cultural, socio-economic, and political specificities of each city [66].

In the short, medium, and long term, studies should be conducted on the actual impact of smart city initiatives based on the triple-helix model and even those that include four or five helixes. In addition, measuring citizen participation and governance is critical. At this point, it would be relevant to study how citizen participation can be strengthened and promoted through indicators based on the triple-helix system [67]. Finally, it is suggested to include ways to address the challenges and opportunities posed by emerging technologies such as artificial intelligence, the Internet of Things, and big data, among others [68].

Furthermore, in the long term, the proposal of a model is expected that contributes to the construction of smart and sustainable cities and/or territories, focusing on management and development of their capacities and potentials. This model would allow for a comprehensive and homogeneous approach, focusing on the integration of citizens in the process of city building and visioning.