Measuring Citizens-Centric Smart City: Development and Validation of Ex-Post Evaluation Framework

Abstract

This research aims to present a standardized evaluation system to review and further enhance users’ levels of satisfaction with technologies, facilities, and services of a modern smart city at a time when the smart city paradigm has shifted from the focus of its infrastructural features to citizens. The study also seeks to verify the standardized system, so as to explore the possibility of its future application. For the goals, this research established the Structural Equation Model (SEM) based upon the basic structure of the Customer Satisfaction Index, which is a widely used ex-post assessment model, and upon implications of related studies. To verify the SEM, this study chose two cities, which are located far away from one another and employ different business methods, and conducted a survey of 212 and 197 residents, respectively, with the results being applied to the model for analysis to ascertain if the SEM is reliable and adequate. The analysis results showed that the model secures explanatory power in statistical terms, partially proving that it can be developed into a post-evaluation system for a citizens-centric smart city down the road. However, as meaningful differences were spotted in accordance with characteristics of each urban project, this study tried to come up with the background information of and reasons for such variations, to present implications for urban planning.

1. Introduction

A smart city used to be defined in the 1990s [1] as a city that theoretically adopts newly invented technologies. However, for the past 20 years, its definition and perspectives have been diversified in line with its academic, technical and spatial applications [2] to better reflect infrastructural, economic and social changes and the subsequent needs. Despite such variation, there exists a consensus in that the smart city aims to promote a sustainable city by advancing it through the introduction and the utilization of Information Technologies (IT), and information and communication technologies (ICT) [3].

Such a consensus could have well been maintained because widely publicized projects for the smart city construction [4] have drawn attention for their introduction of large-scale advanced technology infrastructure in major countries and cities around the world. In particular, the smart city has emerged as a new alternative for resolving the side effects of sudden urbanization in such fields as transportation, water and waste, safety and security, and healthcare [5], on the back of the development of ICT, as well as the sustainable development advocated by multinational institutions [6,7].

As such, the smart city has sometimes served as a barometer of the competitiveness of a country [8] or a city in terms of its sustainability from the outset to date [9]. Accordingly, companies and multinational research institutes that lead the development and the distribution of the aforementioned technologies have devised and applied standardized indices to evaluate the smart city based on major social and economic indicators of various countries and cities [10]. This indicates that a smart city project has been a useful tool to judge the competitiveness of an entity [11] and assess its development potential, as well as its marketing means.

However, despite the overall focus on competitive factors, there have been growing calls in the academic circle for the importance of citizens, who are members of a city and the end-users of technologies [12], rather than the technology itself and its usefulness as a marketing tool. Just as many scholars have mentioned before, including [13,14,15], that the smart city has been highlighted as a means to enhance city dwellers’ quality of life, rather than for bureaucracy, who ultimately consume technology [16]. The authors of [17,18,19] and others also stressed the significance of both tangible and intangible end-products, including technology, facilities, and services (TFS) from citizens’ point of view, while shedding the competitive comparison between technologies adopted by different cities.

It is noteworthy that the first-generation smart cities, armed with new technologies, became outdated, and the time has come to determine how to maintain and manage them; for example, by improving, replacing, or deleting existing technologies. As an alternative, some have stressed opinion gathering from citizens to solve the unequal supply of infrastructures and services of a smart city. However, quantitative research in this field, just as stated before, is still confined to the distribution of technologies among states or cities, or the performance of specific technologies. In other words, the methodology that puts a focus on citizens has not achieved much progress despite the recent emphasis on the people in this discourse, which has served as a stumbling block to drawing practical alternatives to discussing the sustainability of a modern smart city.

Against this backdrop, this study aims to evaluate technology, facilities, and services of an established smart city from citizens’ perspectives and draw implications for smart city-related policy measures in the future, which is expected to be a practical alternative to a sustainable smart city. For the goal, the theoretical review on a citizens-centric ex-post evaluation system, including satisfaction levels, is carried out, and a research model is to be established so as to present a standardized evaluation system for existing smart cities. In addition, a survey is conducted involving citizens of two cities, both of which adopt smart city-related TFS but employ different ways in implementing urban planning projects.

After analyzing the results, the study explores the possibility of whether the evaluation model presented here could serve as a basis for a citizens-centric assessment model in the future. It also states the implications for urban planning policies by interpreting the differences between the two cities in line with their geopolitical locations and business methods.

2. Literature Review

2.1. Paradigm Shift in the Smart City

The smart city was defined in terms of the creation of the urban environments of a city, along with advanced information technologies, such as ICT technology and innovation [20], as well as changes in the urban paradigm of globalization [21]. Then, in the following decade, the concept expanded to include such keywords as integration, citizens, and public services. After it developed further, considering knowledge and community factors, among other things, the smart city gained a new goal of ensuring the wellbeing of citizens and the promotion of their quality of life [22].

More researchers in the field shed have light on smart cities’ socioeconomic aspects, while criticizing current studies on smart cities that have failed to consider citizens as a key factor [23,24,25,26], though they acknowledge that centering on physical infrastructure is easier to draw tangible results.

Recent studies, including [18,27,28], also highlighted the need to devise a citizens-centric smart city and services, as well as to set up separate organizations in charge of its maintenance, and to devise an evaluation system, noting that citizen-centered services cannot exist without the consideration of the citizens who reside within it.

These views contributed to defining the smart city, even in the technical circle, as a consolidated infrastructure that provides various services to citizens in both technological and social terms based on ICT technology [29,30]. Moreover, the trend also led private companies, such as [31,32], to emphasize that urban dwellers are a key element of a smart city, calling them the “smart citizen”. After a series of discussions, a “smart citizen” currently means an individual who consumes services installed in a city or via his or her individual devices, as well as a subject who evaluates and interprets those services and then directly affects the related decision making.

In this regard, the citizens-centric smart city, which is the key focus of the modern smart city discussions, is meant to supply technology, facilities and services for citizens. In addition, the public sector, as well as private firms and organizations, improve ways to produce, supply and sell related goods by monitoring citizens’ consumption patterns and analyzing their feedback. A series of such acts allows those private entities to secure sustainability. Accordingly, citizens are direct consumers of goods in a smart city and, at the same time, are those who produce important information that can be used by suppliers. This paradigm shift indicates that the smart city can be maintained by seeking changes from its top-down development strategies to its bottom-up systems, as well as through proactive communication with smart citizens [33].

Likewise, supplies of technology, facilities and services of a smart city are required to establish a fresh bottom-up system. Various studies including [23,34,35] also suggest that the need is urgent to establish a system to measure and evaluate the value of smart city goods currently used, in order to advance the city from its development and infrastructure construction stage, which is expected to ultimately generate co-creation between users, i.e., citizens and stakeholders.

2.2. Criticism and Emergence of Citizens-Centric Smart City

Over the past decade, the concept of a smart city has changed its focus on infrastructure to a focus on citizens, which brought some projects—both ongoing and completed—under criticism.

For example, Masdar of Abu Dhabi, so called Smart City 1.0, was criticized for lacking sustainability of the project itself [36], and the business model for the city of Songdo in South Korea came under criticism for its employment of the top-down methodology that is frequently used for existing ubiquitous cities, as well as for its failure to achieve stated economic and social goals [37,38]. Other cases have also been cited for their lack of educational programs and services for citizens to induce their practical participation in smart city projects [25]. Such criticism has sparked controversies as well as new implications in two perspectives:

First, such criticism is based on differences in the perception of a smart city in the past and at present. As mentioned above, recent studies see citizens as a key element of the smart city in the era of Smart City 2.0 or above, but the so-called first-generation smart cities, presented from the late 1990s to the early 2000s, were meant to be experimental cities that introduced intelligent infrastructure with a goal to improving their technological and engineering performances. The notion has since undergone changes, such as a digital city, an intelligent city, a ubiquitous city, a knowledge city, and a sustainable city, by taking into consideration both technologies and human beings [39].

Therefore, smart city-based urban development and regeneration projects of the past did not review human (or citizen) factors [40,41] and ended up missing technologies, facilities and services that citizens directly experience and use [42,43], which became a source of criticism.

Second, critics focus on incomplete smart city projects due to their “excessive” future-oriented perspectives. The authors of [44,45] claimed that current smart city projects are concentrating only on the introduction of new technologies while failing to fully consider how to contribute to making a civil society, and to guaranteeing its sustainability. However, they may have overlooked the fact that ongoing smart city projects bear high chances of being modified over the course of their execution. Especially when a project is kicked off from scratch to combine the development of a new town or city with smart city infrastructures, it will take quite a long time before the urban space is equipped with human and material resources to have it actually serve its due role. In addition, changes in financial situations, such as planned budget; urban-, architectural-, and information-related policies; and political circumstances could become a considerable stumbling block to the completion of well-organized smart cities both in internal and external terms, just as many theories and studies have pointed out. In this regard, any criticism against smart city projects should be made based on goals pursued by each city. Its focus also needs to be on whether a city is being created to meet the needs of its actual users.

Taken the above-mentioned points together, the criticism for established smart cities was due largely to different viewpoints among studies. There are limitations to fitting a wide spectrum of notions into a singular notion of an “ideal” smart city. However, those critics are on the same page in that they emphasize the lack of attention to citizens and civil societies [46,47], and call for providing appropriate technologies, facilities, and services for users [48]. In this context, it is necessary to establish evaluation criterion which fully considers citizens, and this will be a way to promote the sustainability of many smart city projects in the future [49]. Studies on citizens who reside in places where smart city projects have already been implemented can be a starting point to achieve the citizens-centric smart city in real terms.

3. Research Methodology and Materials

3.1. Research Framework and Model

It is necessary to listen to citizens, who are end-users of services and the subject of reproducing public information, in order to build a citizens-centric smart city. In the field of academic research, the establishment of a method to systematically perform their opinion gathering is also needed. As [50] stated, there exists a wide range of differences in interpreting a smart city in terms of system, governance, information, space, and design, and even some theories clash. However, in the area of an actual smart city project, the standardization of its technologies, facilities, and services has been called for [51,52].

As such, this study aims to present a standardized research model, which is expected to present the analysis of citizens’ use of related goods, as well as the implications for promoting their continuous utilization when related entities create, operate, and manage a smart city. It also aims to establish an evaluation model based on a widely used ex-post evaluation research model for consumers, and to lay the groundwork for standardizing tools to collect and assess citizens’ opinions on smart city infrastructures. This research will also verify the evaluation model by carrying out surveys and their quantitative analysis based on the population and samples to discuss their future applications.

In order to establish the evaluation model, this study adopted the basic structure of the Customer Satisfaction Index. The structure of this research model differs according to usages by each entity, but this has significance as it would be a tool to explore consumers’ cognition on public or private goods both in the past and at present; their experiences, and the possibility of their continuous use down the road.

Just as shown in Figure 1, major Customer Satisfaction Indexes include ACSI (American Customer Service Index), SCSB (The Swedish Customer Loyalty Barometer), ECSI (European Customer Loyalty Barometer), and NCSI (National Customer Service Index). They are based on the Structural Equation Modeling (SEM), which began to be discussed in the 1970s and was expanded to analyze service quality, or SERVQUAL, in 1980s [53].

These SEMs are used not only to measure corporate products, but also to evaluate the level of satisfaction [54] of both tangible and intangible goods provided by private and public entities as well as their brands. Despite some differences depending on countries, cities and evaluation agencies, the basic SEM structure for each index bears similarities [55].

Looking into details, the variables—“Customer Expectations” with goods, “Perceived Quality” that they evaluate after use of the goods and services, and “perceived value” of the goods for consumers’ expenses—affect the overall and comparative “satisfaction,” and those variables are defined as being interconnected. “Satisfaction” is also linked to “complaint”, which means that users’ satisfaction can be affected by suppliers’ responses when they have problems with the goods. It also indicates their expectations of the responses they receive when they have complaints. “Complaint” can then lead to sustainability, as the handling of complaints would affect consumers’ decisions on whether to continue to use the supplies and/or recommend them to others. In this regard, the SEM structure is expected to help discover the perceived advantages and disadvantages of goods, which consumers felt in the past, feel at present, and will feel in the future, and to draw implications for the improvement of the goods.

In other words, it enables a diachronic point of view in learning the formation of consumers’ cognition regarding goods in the past; their current satisfaction; and a will to use in the future. Each item of the variable is also placed on a Likert scale, which could lead to various implications through the selective analysis of items when necessary, without using the SEM. In terms of standardization, the SEM in particular could show some differences between results shown in advanced nations and in emerging nations, just as [56] pointed out. However, it is noteworthy that the very basic structure is still effectively applicable to diverse industry fields within a country or a city [57]. In this context, if the SEM structure is applied to the evaluation of a citizens-centric smart city and it could secure statistical explanatory power, the smart city can be regarded as a kind of new goods, industry, or service of a country or a city. However, the traditional SEM model was modified here, as follows, to evaluate a smart city from citizens’ point of view, just as shown in Figure 2.

First, the relationship among variable groups was re-established. The SEM for the above-mentioned Customer Satisfaction Index has already been verified through various studies. However, this study specifically considered citizens, who consume technologies, facilities and services of a smart city and produce related information. As such, “User Characteristics (UC)” was added to the modified version, just as shown in A, to see if each respondent’s demographic, economic, and social features can be well reflected to the service evaluation model. In addition, the study directly linked the ‘User Characteristics’ to “user Satisfaction” to determine if citizens’ characteristics affect their levels of satisfaction with a smart city, and if so, how much impact it would have, through which the research can draw implications for related studies on a citizens-centric smart city.

Second, evaluation items to measure the accessibility of a smart city’s TFS from the citizens’ point of view were added and verified. The insertion of these variables is based on revelations that smart city-related services provided in each country, city or region have not been frequently used, in case they are not well recognized by the people due to their low accessibility or usability [58,59]. This concept is similar to “affordance”, a behavioral psychology term, which means a way to define individuals’ behaviors through a series of experiences and knowledge. This study, however, puts a greater value on the accessibility to discuss users’ overall access to a number of goods, rather than on a single item with a clear intention. In fact, smart city-related studies have concentrated on accessibility in multiple aspects, such as its improvement in order to promote citizens’ mobility both indoors and outdoors [60]; the consideration of a greater number of populations to achieve a smart but still inclusive city [59,60]; and the public access to data collected, processed and disclosed through smart city tools.

In terms of the creation of a citizens-centric smart city, no matter how many technologies, facilities and services are adopted, they would ultimately be unsustainable if they fail to actually be utilized. So, smart city-related services are bound to be affected by their accessibility as well as being linked to citizens’ intentions to experience them, and the citizens’ satisfaction. Therefore, this study constructed the SEM by directly connecting “accessibility” with “satisfaction” and “complaint”, just as shown in Figure 2. The variables and items of each group are stated in

Table 1. List of Evaluation Criteria.

L.V.	Index	Items	Class.
User Characteristics (UC)	UCm1	Respondents’ gender	Male/Female
	UCm2	Respondents’ age	Age
	UCm3	Respondents’ income level	Income level
User Expectations (UE)	UEm1	Overall expectations for SC	Likert 11pts.
	UEm2	Expectations for SC demand satisfaction	Likert 11pts.
	UEm3	Expectations for SC service reliability	Likert 11pts.
Perceived Quality (PQ)	PQm1	Satisfaction for overall expectations	Likert 11pts.
	PQm2	Satisfaction of overall demands	Likert 11pts.
	PQm3	Satisfaction of overall reliability	Likert 11pts.
Perceived Value (PV)	PVm1	Price appropriateness compared to SC facilities	Likert 11pts.
	PVm2	Facility appropriateness compared to costs	Likert 11pts.
Accessibility (AC)	Am1	Accessibility to SC’s TFS	Likert 11pts.
	Am2	Readability of SC-TFS guide, signs	Likert 11pts.
Satisfaction (SATIS)	USm1	Overall satisfaction with SC-TFS	Likert 11pts.
	USm2	Satisfaction compared to SC-TFS expectations	Likert 11pts.
	USm3	Satisfaction compared to highest SC-TFS expectation level	Likert 11pts.
Complaint (CP)	SCm1	Dissatisfaction or satisfaction with SC problem solving	Likert 11pts.
	SCm1-2	Dissatisfaction or expectations for SC problem solving	Likert 11pts.
Sustainability (SUS)	SUm1	Reuse intention of SC-TFS	Likert 11pts.
	SUm2	Recommendation intension of SC-TFS	Likert 11pts.

.

Indicators here are similar or identical to those in the above-mentioned consumer satisfaction model, and they are presented in the form of questions to enhance respondents’ understanding. All the items are on the Likert 11 scale, ranging from 0 to 10, and all items are required to be answered. In case of “complaint”, however, the items are designed to assess respondents’ expectation of suppliers’ problem solving because they may not have any complaints with the TFS of a smart city.

Along with the establishment of the evaluation model, this study presented for the questionnaire comprehensive information on the technologies, facilities and services of the smart city established in the target sites, to help respondents better understand what the TFS means, as some might have not recognized them even though they have used the items. So, the list of their TFS was drawn up based upon their administrative information, related reports, and academic documents. Second, interviews were conducted with government officials, scholars, and researchers in the related fields of each city to redefine the TFS currently in operation, as is shown in

Table 2. List of Smart City Technology, Facility, and Service (TFS) of Barcelona and Songdo.

L.V.	Items	List of Smart City Technology, Facility, and Service
Barcelona (A)	Disclosure of public information; Smart education and tourism;Exchanges/Promotion; Urban planning projects; Life/community; Ambulation/traffic monitoring.	Visualization of city operating system and solution energy use/costs; General civil petition/administrative computer processing; CCTV for crime prevention; AWCS (Automated Waste Collecting); smart trash can; smart street light; public WiFi; smart-tech education program; smart city conference; public relations center; smart city tourism; tour guide app; R&D and support center; protection/management of elderly people; location-based local information sharing; public bicycle; car sharing; smart traffic management; real-time bus guide signboard.
Songdo (B)	Data collection and Provision; Smart street furniture; Urban development and construction projects; Safety-, health-, traffic-, environmental-monitoring system.	e-General civil petition/administrative computer processing; integrated living info (parking, medical institutions, tourism, restaurants, and educational institutions, etc.); real-time air quality, water quality, and weather info (traffic/environment/safety/disaster); real-time air quality/weather information; real-time traffic/public transportation info; information-providing kiosks and media boards; public WiFi; real-time CCTV web camera; CCTV for crime prevention; public relations center; R&D and support center; integrated control; emergency care for the elderly living alone; public and shared bicycles; real-time traffic violation control and traffic safety services.

below. Third, the subjects of the survey were confined to those who have had hands-on experience with TFS, so as to increase the objectivity of this research and the verification ability of the evaluation model. In order to make sure that they have used the TFS, photos and other visual materials of the infrastructure were provided to the respondents.

3.2. Study Areas and Data Collection

In addition to the research model presented above, this study conducted a survey of residents and visitors in two cities, which have often been cited as prime examples of modern smart city projects in related studies, so as to evaluate their smart city technologies, facilities and services in a comprehensive manner.

Barcelona (A) is where the TFS projects have been gradually conducted in traditional urban spaces by local governments, while Songdo (B) is where a large-scale new city development project was implemented along with the TFS introduction at the same time. However, as this study is mainly aimed at examining and verifying the applicability of the research model based upon the analysis of citizens’ responses, the name of the cities and the detailed services, advantages and disadvantages of each smart city project, and their satisfaction level will not be described here. Instead, the description will be based on the figures needed to analyze the research model and survey results. Specific research sites in the two cities were chosen through discussions with experts and civil servants in each city in order to secure valid samples. The spatial scope for the survey was set within a 2-km radius of where major TFS projects were carried out. The center of City A is where support facilities for knowledge-based industries were established as part of smart city projects, and the center of City B is where a center for the smart city operation, management and experience is located.

This study also fixed the number of respondents through stratified sampling based on the population of each administrative district and their age. However, the population setting and sampling were limited, to involve those aged between 15 and 65 in Barcelona, and those aged between 20 and 65 for Songdo. Both represent around 65.8 percent of the total population on average. The questionnaire was translated into English and the language used in each nation. The pilot survey was carried out including 52 officials in related fields in Barcelona and 73 in Songdo, in May 2018. Following the revision and review of the questionnaire, the main and additional polls were conducted from July 2018 until January 2019. Responses from 211 out of 421 people (Barcelona) and 197 out of 522 (Songdo) people were used for analysis.

3.3. Methodology

This study adopted the Partial Least Square (PLS) to analyze the SEM. As a data modeling technique for spectral data, PLS performs ordinary least squares (OLS) regressions with the least square algorithm, and it is achieved by extracting from the predictors the latent variables. As it is meant to improve the rationale of the model based on the coefficient measurement that maximizes R², the PLS is suitable for the development and the verification of a theoretical model and is widely used in such fields as behavioral science, marketing, and organization [61].

Given that “User Characteristic” in Figure 2 is not a reflective indicator but a formal indicator, the PLS-SEM is deemed suitable for this research. As an analysis program, SmartPLS ver.3.2 was employed, and the internal consistency, reliability, validity, and the discriminant validity, among others, were applied in a comprehensive manner for its assessment. However, the composition reliability (CR) was mainly used in evaluating the internal consistency reliability of measurement variables, and the equations are as follows.

$$CR\left(P_c\right)=\frac{{\left({{\displaystyle \sum }}_{i=1}^M{L}_i\right)}^2}{{\left({{\displaystyle \sum }}_{i=1}^M{L}_i\right)}^2+{{\displaystyle \sum }}_{i=1}^{M}var\left(e_i\right)}, \frac{{\left({{\displaystyle \sum }}_{i=1}^M{L}_i\right)}^2}{{\left({{\displaystyle \sum }}_{i=1}^M{L}_i\right)}^2+{{\displaystyle \sum }}_{i=1}^M\left(1-L_i^2\right)}$$

(1)

where L_i is the standardized outer loadings of the latent variables, e_i is the error of measurement variable I, var(e_i) is the variance (1 − $L_i^2$) of measurement error, and M is the number of variables.

As the PLS adopted here is mainly used for non-parametric statistics, the t-value was calculated through bootstrapping to verify the significance of each path coefficient. The evaluation criteria to verify the SEM and each variable are shown in

Table 3. List of Criteria for Measuring SEM’s Reliability and Validity.

Criteria	Items	Evaluation Criteria
Reliability	Cronbach’s Alpha	Cronbach’s Alpha ≥ 0.65
	Dijkstra-Henseler’s rho_A (pA)	pA ≥ 0.7
	CR (composite reliability, pc)	pc ≥ 0.6
Convergent Validity	Outer Loading Relevance (L)	L ≥ 0.4 accepted (0.4 ≤ L < 0.7 judgment required)
	Average Variance Extracted (AVE)	AVE ≥ 0.70 (variable removal when falling short)
Discriminant Validity	Fornell-Larcker Criterion (FLC)	Adopted when AVE is higher than the greatest value of correlation
	HTMT (Hetorotrait-Monotrait Ratio)	HTMT ≥ 1 abandonment (lack of Discriminant Validity)
Inner VIF	Collinearity Statistic	VIF ≤ 5 adoption (no Variance Inflation Factor in variables)
Outer VIF	Collinearity Statistic	VIF ≤ 5 adoption (no Variance Inflation Factor in variables)
Bootstrapping	P	p ≤ 0.05 (95%)
	T	t ≥ 1.96 (95%)
	Coefficient of determination (R2)	R2 ≥ 0.50 (moderate)
	Q2	Q2 ≥ 0.00 (cross-validated redundancy)

below.

4. Results

4.1. Overview

In order to examine smart city technologies, facilities and services in the two cities for the evaluation of the SEM, this study conducted a survey on statistically significant samples, and analyzed the results. As shown in

Table 4. Overview of Descriptive Statistics of Respondents in Barcelona and Songdo (n = 408).

Class. 1.	Class. 2.	Barcelona (n = 211/172,780)		Songdo (n = 197/121,988)
Gender	Male	108	51.00%	102	52.00%
	Female	103	49.00%	95	48.00%
Age Group	(15, Barcelona) 20–29	44	21.00%	45	23.00%
	30–39	65	31.00%	53	27.00%
	40–49	51	24.00%	47	24.00%
	50–59	40	19.00%	35	18.00%
	60–65	11	5.00%	16	8.00%
Income Levels (monthly)	Under 150 USD	25	12.00%	16	8.00%
	150–250 USD	32	15.00%	30	15.00%
	250–300 USD	63	30.00%	33	17.00%
	300–400 USD	53	25.00%	83	42.00%
	Over 400 USD	38	18.00%	35	18.00%
Administrative Districts of Sample	District 1	71	33.82%	37	18.83%
	District 2	46	21.82%	61	31.09%
	District 3	55	26.00%	49	25.09%
	District 4	38	18.00%	49	25.00%

, the target site encompassed four administrative districts within a 2-km radius, and all the respondents either resided or stayed in the area. The survey was conducted face-to-face by researchers from both countries.

Reponses made by those who had experienced smart city TFS or were currently using them were chosen for analysis: In Barcelona, 211 out of a total 421 respondents were selected, and 197 out of 522 people were chosen in Songdo. The sampling error stood at ±6.74% (Barcelona, n = 211) and ±6.98% (Songdo, n = 197), at the 95 percent confidence level for 172,780 respondents each.

When asked if they know or have ever heard of a smart city, 73% of respondents (n = 421) in Barcelona (n = 437) and 84% (n = 309) of respondents in Songdo (n = 522) said that they have heard about a smart city, including such terms as digital city and ubiquitous city. However, the number of those who actually used the technologies, facilities and services installed or distributed in a smart city came far lower, at 211 (50%) and 197 (38%), respectively. Despite the low utilization rate of smart city infrastructures in overall terms, the survey showed that each respondent has used or currently uses an average of 5.27 TFSs in Barcelona and 4.12 in Songdo. The mean values for each variable collected according to the aforementioned survey are shown in the

Table 5. Comparison of Mean Values by the Indicators (n = 408).

L.V.	M.V		Barcelona	Songdo
User Expectations (UE)	UEm1	Overall expectation for SC	7.69	7.43
	UEm2	Expectations for SC demand satisfaction	7.63	7.58
	UEm3	Expectations for SC service reliability	7.40	7.66
Perceived Quality (PQ)	PQm1	Satisfaction for overall expectations	7.84	6.78
	PQm2	Satisfaction of overall demands	8.06	7.33
	PQm3	Satisfaction of overall reliability	7.90	6.94
Perceived Value (PV)	PVm1	Price appropriateness compared to SC facilities	6.19	6.77
	PVm2	Facility appropriateness compared to costs	6.87	7.96
Satisfaction (SATIS)	USm1	Overall satisfaction with SC-TFS	8.03	7.68
	USm2	Satisfaction compared to SC-TFS expectations	7.23	6.92
	USm3	Satisfaction compared to highest expectation	6.84	6.21
Accessibility (AC)	Am1	SC-TFS accessibility	8.01	7.04
	Am2	Readability of SC-TFS guides, signs	7.64	6.36
Complaint (CP)	SCm1-2	Expectations for problem solving	7.18	7.11
Sustainability (SUS)	SUm1	SC-TFS reuse intention	7.42	6.60
	SUm2	SC-TFS recommendation intention	7.30	7.30

.

Looking into the details, people in Barcelona turned out to be nearly unaware of such non-exposure services as the city’s comprehensive control centers, and energy and traffic management, despite the authorities’ PR efforts. In the case of Area B, people do not know or utilize the real-time information on air and water quality and weather provided by websites or applications, as well as media boards that provide local news and information, and the safety management service for elderly citizens living alone, according to the survey. The results could have been caused by different types of TFS introduced in each city, their arrangement, PR campaigns, differences in local culture, and citizens’ way of life. These factors also can create different results in different regions, even though the same or similar TFS are installed.

4.2. Review of Research Model

In order to evaluate technologies, facilities and services of a smart city from the citizens’ point of view, this research, using the SEM and the results of the survey on each item, was applied to the model to assess their suitability and generality. The verification of this SEM was made in two phases.

First, the entirety of the samples from Barcelona and Songdo were put to the SEM to check their suitability. Second, samples of each city were inserted for analysis. If differences are spotted between the cities, the research will analyze their causes to discuss ways for improvement. If the analysis results are similar to the modeling of the existing Customer Satisfaction Index, this SEM can be said to prove its value, at least in part, as a standardized model for a citizens-centric smart city in the future. If this model that deals with multiple technologies, facilities and services of a smart city is statistically significant, it means that it can serve as a tool to evaluate users’ satisfaction by regarding multiple TFSs as one smart city, as well as to measure their evaluation on each service or technology at the same time.

4.2.1. Review of Research Model

In this part, the survey results of all respondents (n = 408) in both cities were inputted. As mentioned above, the convergence validity of the SEM was determined based on the outer loadings (L_i), the reliability of the measurement variables ($L_i^2$), and the Average Variance Extracted, or AVE.

According to the analysis of the initial model shown in

Table 6. Convergent Validity Result of Barcelona and Songdo (n = 408).

L.V.	M.V		Outer Loading	Reliability	Ave
			Over 0.70	Over 0.50	Over 0.50
UserCharacteristics (UC)	UCm1	Gender of respondent	0.756	0.572	0.775
	UCm2	Age of respondent	0.774	0.599
	UCm3	Income level of respondent	0.789	0.547
User Expectations (UE)	UEm1	Overall expectation for SC	0.880	0.774	0.789
	UEm2	Expectations for SC demand satisfaction	0.896	0.803
Perceived Quality (PQ)	PQm1	Satisfaction for overall expectations	0.870	0.757	0.695
	PQm2	Satisfaction of overall demands	0.799	0.638
	PQm3	Satisfaction of overall reliability	0.830	0.689
Perceived Value (PV)	PVm1	Price appropriateness compared to SC facilities	0.917	0.841	0.864
	PVm2	Facility appropriateness compared to costs	0.942	0.887
Satisfaction (SATIS)	USm1	Overall satisfaction with SC-TFS	0.778	0.605	0.684
	USm2	Satisfaction compared to SC-TFS expectations	0.853	0.728
	USm3	Satisfaction compared to highest expectation	0.848	0.719
Accessibility (AC)	Am1	SC-TFS accessibility	0.899	0.808	0.758
	Am2	Readability of SC-TFS guide, signs	0.841	0.707
Complaint (CP)	SCm1-2	Expectations for problem solving	1.000	1.000	1.000
Sustainability (SUS)	SUm1	SC-TFS reuse intention	0.863	0.745	0.795
	SUm2	SC-TFS recommendation intention	0.919	0.845

, the outer loadings (L_i) came below 0.5 in case of “the expectation level for reliability”, which corresponds to variable No. 3 within “user expectation.” Accordingly, this research excluded the variable from the research model, and carried out the analysis again, which led to the results that the outer loading value stood at 0.07 or higher, and therefore proved its validity and reliability. The analysis of reliability $L_i^2$ showed the level of 0.5 or higher to guarantee its credibility, and the AVE value also comes to 0.5 or higher to secure the reliability of the individual variables and the validity of latent variables.

The internal reliability evaluation of each variable is shown in

Table 7. Result of Reliability-Validity of SEM’s Latent Variables (Barcelona and Songdo, n = 408).

L.V.	Cronbach’s Alpha	Dijkstra-Henseler’s rho_A(pA)	CR	Heterotrait-Monotrait Ratio (HTMT) 1
	(Over 0.60)	(Over 0.70)	(Over 0.60)	(Not Includes 1)
User Characteristics	-	0.705	0.915	Y
User Expectation	0.732	0.734	0.911	Y
Perceived Quality	0.780	0.789	0.881	Y
Perceived Value	0.844	0.863	0.937	Y
Satisfaction	0.770	0.781	0.869	Y
Accessibility	0.684	0.705	0.766	Y
Complaint	0.997	0.991	0.996	Y
Sustainability	0.745	0.866	0.915	Y

¹ Not includes 1 in Confidence Interval.

, and the Cronbach’s Alpha, rho_A and CR values came to 0.6, 0.7, and 0.6 or higher, respectively, indicating that all potential variables have internal reliability. Moreover, the results of the discriminant validity by Fornell-Larker and the Chi-square test also turned out to be suitable, though they are not presented in this research. As the cross-validity of latent variables by HTMT criteria did not include one within the 90% confidence interval, each variable can be said to secure the discriminant validity. As such, the SEM proposed in this study secured consistency, reliability and validity for each item. Then, the PLS-SEM algorithm and bootstrapping were executed to analyze the SEM and derive the outer loading (regression coefficient).

As shown in

Table 8. Result of Coefficient of Determination on Latent Variables (based on R2 and Adj. R2).

L.V.	R2	Adjusted R2
User Expectation	0.142	0.14
Perceived Quality	0.531	0.529
Perceived Value	0.261	0.258
Satisfaction	0.697	0.694
Complaint	0.309	0.307
Sustainability	0.414	0.411

, “satisfaction” revealed the highest explanatory power (0.694) based on the modified R2 criteria, which means that this research model is best optimized for measuring the actual satisfaction level. In addition, the VIF values of all variables in this SEM came to less than five, and no inter-multicollinearity occurred. Each measuring variable included in the structural model also came to less than five, indicating that there was no inter-multicollinearity.

As shown in

Table 9. Result of Paths on a Pair of Latent Variables (Barcelona and Songdo, n = 408).

Path	Original Sample	Mean.	STD.EV	t Stat.	p	VIF
Cognition→Satisfaction	0.324	0.324	0.043	7.505	0.000 **	1.924
Complaint→Sustainability	0.139	0.139	0.042	3.333	0.001 *	1.000
Perceived Quality→Perceived Value	0.203	0.203	0.069	2.949	0.003 *	2.071
Perceived Quality→Satisfaction	0.131	0.132	0.048	2.750	0.006 **	2.559
Perceived Value→Satisfaction	0.262	0.263	0.049	5.373	0.000 **	1.363
User Characteristics→Perceived Quality	0.128	0.13	0.037	3.500	0.000 **	1.165
User Characteristics→Satisfaction	0.173	0.171	0.035	4.929	0.000 **	1.223
User Characteristics→User Expectation	0.377	0.381	0.047	8.077	0.000 **	1.002
Satisfaction→Complaint	0.556	0.556	0.041	13.419	0.000 **	1.000
Satisfaction→Sustainability	0.556	0.557	0.054	10.235	0.000 **	1.447
User Expectation→Perceived Quality	0.671	0.671	0.036	18.89	0.000 **	1.165
User Expectation→Perceived Value	0.345	0.346	0.065	5.335	0.000 **	2.071
User Expectation→Satisfaction	0.212	0.211	0.051	4.172	0.000 **	2.381

** p < 0.000, * p < 0.05.

, all the single paths presented in this study model were found to have explanatory power within the scope of being statistically accepted. When setting the respondents’ satisfaction as a final path, the highest value (based on t-stat.) turned out to be “cognition” of technologies, facilities, and services of a smart city, which was emphasized by this study; this was followed by “Perceived Value (PV)”, “User Characteristics (UC)” and “User Expectation (UE)”.

That is, citizens’ satisfaction can be maximized by the fair level of cognition of and accessibility to the smart city elements (t = 7.505, p = 0.000). This also indicates that the value of each TFS that citizens think of for their expenditure (t = 5.373, p = 0.000), socioeconomic features such as age, gender, and income level of citizens (t = 4. 929, p = 0.000) and their overall expectation level with a smart city (t = 4.472, p = 0.000), can lead to their higher satisfaction. These results suggest that there is a need to better consider such factors as citizens’ age, gender, and income (based on questions about how much each smart city infrastructure is exposed to citizens), when drawing a plan regarding smart city elements from citizens’ perspectives. It also means that, if charging for those technologies, facilities and services are needed, and economic reviews on whether the price was affordable for the current residents are needed. It is safe to say that the level of satisfaction formed through this consideration would lead to their continuous use and utilization of smart city materials (t = 4.929, p = 0.000) based on its reliability (t = 13.419, p = 0.000).

4.2.2. Analysis of All Paths of Research Model (Barcelona and Songdo)

The individual paths are determined by the mediating effect of each variable inserted into the structural model. The path model presented by this research as well as the value and the flow of the path set by 5000 times of bootstrapping are shown below. As previously mentioned, this research model is based on the SEM that evaluates images of traditional goods, services, or their suppliers, and it involves users’ expectations for smart city elements, perceived quality and value of goods after use, and chances of their future utilization based on the level of satisfaction. In this regard, as shown in Figure 3, this research model proves that technologies, facilities and services of a smart city can be considered as traditional goods and services and they can be evaluated from the users’ viewpoint.

There exists a lot of research models, paths and analysis methods presented by bootstrapping. As the latent variable in each index group shows the generalized representation in accordance with the increase in the value, however, this study is to take the upper value of each index group and their index variables which affect the satisfaction for interpretation. First, each citizen’s characteristics affect their satisfaction with the TFS, with the impacts becoming greater according to their gender, followed by age and income level. Second, (1) user expectation, (2) perceived quality after use, and (3) perceived value based upon the perceived quality all affect their satisfaction, but the perceived value, which combines (1) and (2) turned out to have greater impacts on users’ satisfaction. That is, the perceived value, which measures how appropriate their payment for those smart city elements was, impacts on their satisfaction the most. Third, “Accessibility” to each facility and service directly affects users’ overall satisfaction. Access directly to goods when necessary contributes more to the improvement of their satisfaction than any other indirect methods, such as promotion campaigns. Fourth, users’ intention of continuing the use of the TFS can be boosted when the goods were better than what users expected. Given that the installation of smart city infrastructure is still in the early stage, and many see chances of its development further, it is crucial to improve the quality of the TFS beyond people’s expectations, rather than focusing on quantitative satisfaction.

4.3. Analysis Results of Research Model for Each City

When the analysis results of the two cities were combined and then inserted into the research model, it was identical to the initial model presented by this study. However, as mentioned in the beginning, o this structural model may reveal differences when the results of each city are separately put into the research model to consider the fact that there are differences in their business methods.

In particular, it is necessary to note that the above-mentioned various Customer Satisfaction Indexes are similar in their basic structure, but have been modified and sometimes improved to fit for each country or an institution. So, the chances are that, considering business characteristics of urban planning projects or economic and cultural differences caused by their geopolitical locations, the basic model proposed above may be different among cities. Accordingly, in this section, the two subjects were analyzed in the same way through the execution of bootstrapping based on the SEM used before.

In the analysis process, based on the evaluation results of the significance and the suitability of each path presented by the bootstrapping results, the low value among the variables within the statistically insignificant latent variable group was first removed. In case the result was not significant, the same process was repeatedly executed—after removing the latent variable group and re-executing bootstrapping—until significant results were derived. The results and differences of the two cities using this process are as follows.

4.3.1. Evaluation and Comparison of Measurement Model

Through the above-mentioned process, analysis continued by roving variables until the statistical suitability was obtained. One variable group (LV) was removed from each research model of the two cities, and the final verification results for the validity showed that both cities show higher values in the outer loading (L_i), reliability ($L_i^2$) and AVE (Average Variance Extracted) than the average. In regard to the details, all measurement items in the “Perceived Quality” of Barcelona were removed as they failed to explain the path of the SEM, and all items in the “user characteristics” in Songdo were taken out due to their failure to explain the path of the structural model.

After excluding the variable group that does not match the research model as shown in

Table 10. Convergent Validity Result of Barcelona and Songdo (n = 408).

L.V.	Barcelona			Area B
	Outer Loading (Li)	Reliability (${\mathit{L}}_{\mathit{i}}^2$)	Ave.	Outer Loading (Li)	Reliability (${\mathit{L}}_{\mathit{i}}^2$)	Ave
	>0.70	>0.50	>0.50	>0.70	>0.50	>0.50
User Characteristics (UC)	−0.719	0.517	0.991	-	-	-
	0.748	0.560		-	-
	0.700	0.490		-	-
User Expectations (UE)	0.879	0.773	0.802	0.876	0.767	0.782
	0.912	0.832		0.892	0.796
Perceived Quality (PQ)	-	-	-	0.862	0.743	0.681
	-	-		0.791	0.626
	-	-		0.822	0.676
Perceived Value (PV)	0.932	0.869	0.889	0.898	0.806	0.837
	0.954	0.910		0.931	0.867
Satisfaction (SATIS)	0.838	0.702	0.731	0.675	0.456	0.619
	0.862	0.743		0.838	0.702
	0.864	0.746		0.835	0.697
Accessibility (AC)	0.897	0.805	0.728	0.896	0.803	0.720
	0.826	0.682		0.798	0.637
Complaint (CP)	1.000	1.000	-	1.000	1.000	1.000
Sustainability (SUS)	0.838	0.702	0.832	0.797	0.635	0.743
	0.862	0.743		0.922	0.850

, the review of the reliability and discriminant validity (HTMT) of the measurement indicators for the results of the two cities were conducted, which showed that the receptive reliability and the validity were secured in most items. In the case of Songdo, Rho_A, an item for evaluating the reliability in the Accessibility group, came below the standard value. However, it still satisfied the CR value (0.6 or more) which is the baseline of the representative reliability set by this study. It also exceeded the standard value in the calculation of Cronbach’s Alpha value (0.6 or more) and HTMT (90% confidence interval 1), which are deemed to have the strictest explanatory power, so that it was determined to be acceptable (see

Table 11. Result of Reliability and Validity of SEM’s Latent Variables.

Area	Criteria	Std.	UC	UE	PQ	PV	SATIS	AC	CP	SU
Barcelona	Cronbach’s α	>0.6	0.662	0.754	-	0.877	0.817	0.629	1.000	0.798
	rho_A(pA)	>0.70	0.710	0.766	-	0.877	0.826	0.640	1.000	0.807
	CR	>0.60	0.766	0.823	-	0.969	0.891	0.689	1.000	0.861
	HTMT	Not include 1	Y	Y	-	Y	Y	Y	Y	Y
Songdo	Cronbach’s α	>0.6	-	0.722	0.766	0.808	0.692	0.619	1.000	0.668
	rho_A(pA)	>0.70	-	0.724	0.777	0.828	0.719	0.703	1.000	0.724
	CR	>0.60	-	0.71	0.823	0.913	0.767	0.759	1.000	0.796
	HTMT	Not include 1	-	Y	Y	Y	Y	Y	Y	Y

).

After removing groups with a low explanatory power and verifying the consistency and reliability of each city, this study conducted bootstrapping 5000 times with the results, and analyzed the structural model and then derived the outer loading (standardized regression coefficient) as follows.

Here, as shown in

Table 12. Results of Each Model’s Coefficient of Determination on Latent Variables (based on R² and Adj. R²).

L.V.	Barcelona		Songdo
	R2	Adjusted R2	R2	Adjusted R2
User Expectation (UE)	-	-	0.505	0.503
Perceived Value (PV)	0.274	0.270	0.367	0.361
Perceived Quality (PQ)	0.186	0.182	-	-
Satisfaction (SATIS)	0.750	0.745	0.627	0.620
Complaint (CP)	0.373	0.370	0.262	0.258
Sustainability (SUS)	0.453	0.448	0.414	0.411

, the approach is a control variable of the “Satisfaction” so that it was not included in calculating the regression coefficient. The analysis results showed that the satisfaction has the highest explanatory power for both models in terms of R² standards, proving that both research models are suitable for measuring the satisfaction of technologies, facilities and services of a smart city. In addition, the value of multicollinearity (variance inflation factor, or VIF) of all variables included in the SEM of the two cities came between 1.000 and 2.555 to hover below 5, which was an acceptable result because each variable was not highly correlated with others

The suitability of each single path is as shown in the

Table 13. Results of Paths on a Pair of Latent Variables (Barcelona).

Path	Original Sample	Mean.	STD.EV	t Stat.	P	VIF
Cognition→Satisfaction	0.321	0.325	0.057	5.614	0.000 **	1.957
Complaint→Sustainability	0.136	0.134	0.048	2.831	0.005 *	1.595
Perceived Value→Satisfaction	0.238	0.240	0.064	3.707	0.000 **	1.279
User Characteristics→Satisfaction	0.212	0.212	0.050	4.219	0.000 **	1.538
User Characteristics→User Expectation	0.523	0.529	0.044	11.914	0.000 **	1.000
Satisfaction→Complaint	0.611	0.608	0.054	11.302	0.000 **	-
Satisfaction→Sustainability	0.582	0.584	0.077	7.548	0.000 **	2.179
User Expectation→Perceived Value	0.432	0.433	0.084	5.132	0.000 **	1.000
User Expectation→Satisfaction	0.325	0.319	0.069	4.742	0.000 **	1.934

** p < 0.000, * p < 0.05.

and

Table 14. Results of Paths on a Pair of Latent Variables (Songdo).

Path	Original Sample	Mean.	STD.EV	t Stat.	P	VIF
Cognition→Satisfaction	0.316	0.318	0.073	4.297	0.000 **	2.072
Complaint→Sustainability	0.161	0.161	0.071	2.272	0.023 *	1.355
Perceived Quality→Perceived Value	0.338	0.337	0.09	3.752	0.000 **	2.021
Perceived Quality→Satisfaction	0.162	0.166	0.076	2.123	0.034 *	2.338
Perceived Value→Satisfaction	0.294	0.297	0.079	3.714	0.000 **	1.589
Satisfaction→Complaint	0.512	0.511	0.06	8.477	0.000 **	-
Satisfaction→Sustainability	0.497	0.5	0.077	6.492	0.000 **	1.355
User Expectation→Perceived Quality	0.711	0.714	0.047	15.035	0.000 **	1.000
User Expectation→Perceived Value	0.318	0.319	0.094	3.395	0.001 *	2.021
User Expectation→Satisfaction	0.179	0.172	0.090	1.990	0.047 *	2.645

** p < 0.000, * p < 0.05.

, and all the paths were found to be statistically significant, and no multicollinearity problem was generated, which guarantee the explanatory power for each path. Similarities and differences drawn from the results between the two cities in terms of each independent path are as follows. First, in the single route that ends with the satisfaction, accessibility has the greatest impacts on the satisfaction in both cities, which is the same result as the analysis of their converged data. This reaffirms that the TFS of a smart city will be able to boost the satisfaction of users or consumers when they are properly placed at a location they want or appropriately guided.

Second, in the single path of Barcelona, the relations between socioeconomic characteristics of citizens (t = 11.914, p = 0.000) and their satisfaction, as well as the satisfaction and expectation for resolving complaints (t = 11.302, p = 0.000), showed quite high values. That is, given that citizens’ age, gender, and income levels affect the satisfaction level, it is crucial to introduce technologies, facilities and services that best fit for citizens’ characteristics in case of established cities. It also can be inferred that listening to users’ complaints and providing a swift resolution could enhance the sustainability of a smart city from citizens’ perspective. Therefore, smart city-related technologies, facilities and services in established cities should consider the current status of the urban space and features of citizens, such as their age, gender, and income level, and it is advised to supply fresh technologies, facilities and services, which proactively reflect the citizens’ opinions. This implies the need for the comprehensive consideration of both physical and social factors of a city, including its form, related policy measures, and unique characteristics.

Third, in the single path of Songdo, the path between user expectation and perceived quality (t = 15.035, p = 0.000) showed a particularly high value compared to others. This indicated that perceived quality, which means the level of satisfaction of individuals’ expectations, is crucial in Songdo, in contrast to Barcelona where the factor was excluded. Citizens’ personal features are important in both areas, but their current social and economic circumstances turned out to specifically affect the satisfaction in Barcelona, while users’ judgments on individual values is the decisive factor in Songdo.

In comparison to Barcelona, Songdo excludes the demographic and economic features of each citizen from the structural model. What this suggests is that a project to build a new city needs to identify a wider range of demands expected to be made by an unspecified number of people, and to prioritize advanced functions, design, and the usability of advanced smart city infrastructure.

4.3.2. Analysis Results of Research Model Routes

The value of an individual path is determined by the mediating effect of each variable inserted into the structural model. The results of the two cities, as shown in Figure 4 and Figure 5, which are summarized in accordance with the path model presented here, and the value and the path flow determined by 5000 times of bootstrapping, are as follows. As explained in the survey results involving the two cities, the level of satisfaction with smart city elements in both cases is set by users’ expectations before their usage and their perceived value after the experience, based upon accessibility, and the satisfaction has a series of paths affecting the possibility of their continued use. This highlights the need for the TFS of a smart city to be considered as just the same as traditional public goods, and from the perspective of users, i.e., citizens.

However, the two cities showed structural differences. First, the satisfaction in Barcelona, where the TFS were introduced in existing urban spaces, was decided by users’ expectations before their firsthand use and their perceived value after the experience, which was affected by individual’s characteristics. In the case of Songdo, where a new city development project is under way, individual user’s characteristics are not factors to be considered, but the quality of the goods evaluated after use leads to their perceived value and satisfaction. However, given that perceived quality is not suitable for the structural model in the case of Barcelona, smart city infrastructures in an established city are needed to be chosen after taking into consideration the features of each citizen, because their expectations for the goods and their value in comparison with users’ payment could boost their satisfaction in such an established city.

Among major variables, first, in terms of users’ expectation, the importance of the reliability of smart city infrastructures was emphasized in both Barcelona and Songdo. Reliability is expected to improve their level of satisfaction with their expenditure for the goods. Second, the perceived value, which is decided after their experiences, was significantly higher in both cities in terms of “the appropriateness of the price set for facilities” than “the appropriateness of facilities compared to costs.” This could indicate that individuals may drop their intention to use the smart city facilities and services in case the price set is higher than the amount they are willing to pay. So, any decision to charge goods or to add additional charges would be required to be made carefully, in order that they do not incur the improvement of the goods.

Third, “overall satisfaction” is the most important factor, among its three items, for technologies, facilities, and services for a smart city to be introduced to existing urban spaces, such as Barcelona. However, in the case of newly built cities, such as Songdo, their satisfaction “compared to the best level of the goods” was higher than their overall level of satisfaction or previous expectations. This result implies that if this research model is verified through further studies as a tool to compare a new city with an established one, the quality improvement in the form, design, and service of the smart city elements could enhance users’ satisfaction, as it will better reflect citizens’ expectations.

5. Implication and Discussion

This study aims to monitor users’ level of satisfaction with the technologies, facilities, and services of a smart city at a time when the paradigm and the focus of a modern smart city has been changing from technology to citizens. It also aims to present a standardized evaluation system and verify it, so as to explore the possibility of its application down the road. For this purpose, the Structural Equation Model was established based on the basic structural model of the Customer Satisfaction Index, which has been widely used across the world, as well as on recent theories. A survey was also conducted involving citizens in two cities to compare their results. In order to draw implications in terms of urban planning, analysis results of the two cities were compared: one is where a smart city infrastructure was established in an existing urban space, and the other is where a smart city project was implemented in a newly built town.

As a result, the combined results of the examination of Songdo and Barcelona showed the identical structure to the prediction model based on the Customer Satisfaction Index. In other words, from citizens’ point of view, the level of satisfaction with a smart city is affected by users’ expectations for its technologies, facilities, and services before having first-hand experiences, their perceived quality after use, cost of the goods, and the accessibility to each facility. Those factors, as well as individuals’ socio-economic characteristics, were found to have guaranteed the possibility of their continuous use. The results indicate that it is important to prioritize the introduction of facilities that residents want in order to achieve a citizens-centric smart city, just as highlighted in many studies. Such a phased introduction is advised in accordance with each city’s demographic, social and economic features, rather than applying advanced technology-based platforms competitively in a drastic manner, as was called for by some previous studies and companies.

Second, the results of the analysis of each city through the research model revealed differences from the basic model, just as forecasted, but their structural features bore similarities. In Barcelona, each citizen’s socio-economic characteristics well explained their satisfaction with smart city infrastructures and their sustainability, while “Perceived Quality” was not suitable for the model. This means that characteristics of each dweller and their expectations can affect satisfaction. In the case of Songdo, where a new city development and construction project takes place, city members’ socio-economic features do not fit for the model. Instead, the reliability of the goods, and moves to meet their personal expectations, were found to boost their satisfaction. From urban planning perspectives, these results indicate that that it is important to adopt credible and high-quality technologies, facilities, and services for a smart city, given the influx of unspecified people, while affordability for existing residents is key to effectively enhancing their overall satisfaction, just as many studies, such as [62,63], pointed out.

This study, however, has limitations in that the comparison of two cities, rather than multiple ones, could not ensure cited academic implications, so follow-up studies are deemed required by involving more entities of a similar scale, form, and conditions to expand this discourse. Many previous studies point to differences among smart cities in accordance with different urban spaces and business methods. At a time when discussions on the establishment of a citizens-centric smart city have expanded, it is crucial to spot differences among the technologies, facilities, and services of a smart city over the course of urban regeneration, redevelopment, and restoration projects, and to make suggestions to come up with proper policy measures suitable for each project [64]. Further requirements are the establishment and/or the revision of related legislation and systems in public sector, such as an opinion gathering system for planning, a monitoring and ex-post evaluation system focusing on citizens, and a certification and evaluation mechanism to maximize the application of those results to wider projects such as a national land planning. In order to create and manage a ‘Sustainable Smart City’, it is also necessary to extend financial support for the private companies or organizations which produce and distribute the smart city technologies, facilities, and services, as well as the expansion of partnerships between public and private entities, among other things [65]. This is the way to Smart City 2.0 and above, where citizens can actively participate and experience, going beyond Smart City 1.0, where citizens were limited to passive participation by technocracy.

In this regard, the research carries significance as it examined the standardized framework to evaluate smart cities from the citizens’ point of view and carried out their basic verification to highlight that smart city infrastructures can be seen as a new commodity of a city and, at the same time, a means of improving the quality of citizens’ lives. Moreover, this study has shed light on the point that a smart city would not be a matter of international competition but a useful way of improving the sustainability of a city or a region, which could be further examined by additional studies down the road.