How to Evaluate Smart Cities’ Construction? A Comparison of Chinese Smart City Evaluation Methods Based on PSF

Abstract

With the rapid development of smart cities in the world, research relating to smart city evaluation has become a new research hotspot in academia. However, there are general problems of cognitive deprivation, lack of planning experience, and low level of coordination in smart cities construction. It is necessary for us to develop a set of scientific, reasonable, and effective evaluation index systems and evaluation models to analyze the development degree of urban wisdom. Based on the theory of the urban system, we established a comprehensive evaluation index system for urban intelligent development based on the people-oriented, city-system, and resources-flow (PSF) evaluation model. According to the characteristics of the comprehensive evaluation index system of urban intelligent development, the analytic hierarchy process (AHP) combined with the experts’ opinions determine the index weight of this system. We adopted the neural network model to construct the corresponding comprehensive evaluation model to characterize the non-linear characteristics of the comprehensive evaluation indexes system, thus to quantitatively quantify the comprehensive evaluation indexes of urban intelligent development. Finally, we used the AHP, AHP-BP (Back Propagation), and AHP-ELM (Extreme Learning Machine) models to evaluate the intelligent development level of 151 cities in China, and compared them from the perspective of model accuracy and time cost. The final simulation results show that the AHP-ELM model is the best evaluation model.

1. Introduction

The globalization trend has greatly expanded the dimensions and populations of cities in recent years, and city planning and coordination has also been improved in a pluralistic way [1]. During this period, problems and contradictions of urban development have been increasing as the days have passed. Several issues have largely curbed the sustainable and harmonious development of cities, such as the aging of the population, the insufficient of per capita resources allocation, the urban traffic congestion, the decline of urban operation efficiency, the backward environmental governance, and the destruction of urban ecology and so on. Along with the development trend of cities and the need to urgently solve the contradictions, the management and development of a good city not only need the help of the traditional city management measures, but also require new technology which can be more scientific and harmonious [2,3].

Innovation and the development of new communication technologies such as Internet of things (IoTs), big data, and cloud computing have provided new solutions to cities’ governance and maintenance. They have built a new model for new cities’ construction [4,5,6]. Integration of information and industrialization have made traditional cities gradually evolve into a new form of ecological social organization. The smart city is based on this trend, and traditional industrial society is transiting to modern innovation society. It can be said that a smart city provides a reliable solution to this process. A smart city gives an optimal approach for a resource-conserving and environmentally-friendly society. From a technical point of view, smart cities can realize the computing and integration of a ubiquitous network, overall perception, and connectivity through the comprehensive application of IoTs, big data, augmented reality, and computing cloud technology. From the perspective of social evolution, the intelligent construction of a city is also needed through the Living Lab, spatial information grid, and data grid integrated application tools and methods, so as to realize background knowledge and open vision, thus coordinating the sustainable development of social form innovation.

In recent years, the global smart city has entered the stage of high-speed development, and it will gradually realize the vision of “total connectivity, comprehensive perception, and intelligence”. The sixth world smart city conference kicked off in Barcelona in 2016 [7], with delegations from more than 600 cities all over the world, and numerous enterprises and experts in communications participated in it. It predicted that by the end of 2016, global smart cities will be worth up to 40 billion US dollars. Additionally, there will be more than 20 countries enacting smart cities’ development planning policies by 2017. They will identify the investment project priority system matching with city information, and prioritize related information technology and the business process. The Prospective Research Report [8] indicated that there were 311 prefecture-level cities and cities in China working on smart city construction by the end of 2015. A total of 158 smart cities have been built and widely used and promoted in more than 70 areas. Therefore, at the planning level, China attaches great importance to the construction and development of smart cities.

In order to solve the problems of supply shortage and the urban infrastructure demand, lack of information resources standardization, insufficient network information security, lag of governance mode of urban government, and lack of technology in the industry driving effect, etc., our paper will summarize the origin and connotation of the smart cities, the urban systems theory, and the people-oriented, city-system, and resources-flow (PSF) evaluation theory. Combining the development characteristics of China’s smart cities, the comprehensive evaluation index system for urban intelligent development is established. Based on the analytic hierarchy process (AHP) and Back Propagation (BP) neural network theory, an evaluation model of the intelligent development of 151 cities in China is evaluated. Finally, we compared the model precision and time cost. This will help to find the bottleneck in smart cities’ construction and provide an effective basis for scientific measures of the development of urban wisdom.

Literature Review

With the development of the smart city concept, research on the connotation and development of the smart city is increasing significantly. The two groundbreaking theories of Graham and Pomeroy, S. M. [9,10] in the 1990s laid the foundation for smart city theory. Previous studies have generally considered urban infrastructure construction as the most important factor. In addition to building, transportation, and other physical facilities, information technology has also dominated the fundamental functions of the city. The research explained how information communication technology influences the development of cities. The introduction of information technology to the evaluation of smart cities is an inevitable requirement of technological development. It is also an important turning point in urban evaluation.

Since then, Allwinkle [11] made a comparative study of Graham and Mitchell. The IBM [3], Forrester Research, Natural Resources Defense Council (NRDC), and the European Intelligence Council, from a different perspective, have given the definition of a smart city [5]. These definitions are highly consistent, that is, smart cities rely on social, public, information, and commercial infrastructure of the city to promote the construction of smart cities through the circulation of resources [12].

Evaluation of the city can support investors as an important guide for the cities to judge their strengths and to define their strategies for future development. Scholars take a different approach to intelligent city assessment. Giffinger R. [13] summarizes the evaluation system as the economy, people, governance, mobility, environment, and living. However, for different cities, there is a need to combine their own characteristics and the analysis of regional diversity [6]. Since then, Etzkowitz H. [14] has comprehensively considered the triple helix of University-Industry-Government. In this way, the intent of urban evaluation is not just to apply the index system, but to find the relationship among social groups. Deakin M. proposed the theory and method to evaluate the correlation of wisdom urban. He chose the triple helix theory to construct the dynamic space of a regional innovation system [15,16,17,18]. On that basis, he reflected on the governance of smart cities, and went on to explain smart cities in terms of the social networks, cultural attributes, and environmental capacities [19,20,21]. Lombardi P. also used the Analytic Network Process (ANP) method to evaluate European cities based on the theory of triple helix. Moreover, he integrated social relations into the evaluation system, and presented an advanced triple helix network model [22,23].

Countries around the world have put forward the related methods of urban construction evaluation. In the 1950s, the United States and Japan began to study a city informatization level evaluation system [24]. The model of Machlup, Borat, the information index, and Information Utilization Potential is a typical method to evaluate urban informatization [25]. Influenced by the development of information technology and Internet of things, IBM first proposed the concept of an Intelligent earth, and cisco put forward the concept of Global Intelligent. There are many international universities that are also involved in the intelligence development study of the index system, such as the University of California, Vienna University of Technology, and the University of Ljubljana, etc. [26]. The global information technology report, jointly released by the World Economic Forum (WEF) [27] and INSEAD [28], introduces the network readiness index (NRI) [29], which provides a basic methodology that has become the most authoritative benchmark preparation tool in information technology reports. In response to the impact of ICT technology, Japan has formulated the u-japan strategy [30] and compared the information of the world’s major countries based on the international assessment of ICT infrastructure construction. In September 2001, the leading company in the field of electronic communications, Kang Sajige, established an evaluation index system that basically covered all levels of smart cities.

In addition, there are many related researches on intelligent city evaluation models and system constructions. Richard Florida [31] and Komninos N. [32] evaluated smart cities from three perspectives. The first is the wisdom of the working population city level, innovation ability, and creativity; The second is associated with the collective wisdom of urban residents, the collective wisdom by differentiation, forming a more creative collaboration competition mechanism; The third is the review of the artificial intelligence embedded city: communication technology infrastructure, digital information space, and provide citizens with online services, etc.

2. Evaluation Index System of Smart City

2.1. Evaluation Index System Based on PSF Evaluation Model

2.1.1. PSF Evaluation Model

The national center for the interconnectivity intelligent city has developed a PSF evaluation model [33] based on the nature of the operating system of a smart city, and it accurately describes the entire idea of the smart city system and its architecture. The three letters of PSF represent People-Oriented, City-System, and Resource-Flow. This model of Resource-Flow (F) from bottom to top corresponds to the input support layer, City-System (S) corresponds to the application layer, and People-Oriented (P) corresponds to the core target layer. Further details are:

(1)

Input support layer, corresponding to resource flow. It covers the infrastructure, the service platform, the flow of resources, the flow of funds, and so on, to meet the flow of resources and exchange, thus providing important material for the urban wisdom and development process.

(2)

System application layer, corresponding to the urban system. Mainly includes environmental protection, urban planning management operations, seamless link wisdom industry development, social system, and intelligent application links such as input and output analysis. It is based on the emergence of large data, such as deep learning technology, providing personalized, customized services for the city, and urban economy to communication, management, service, security, and so on.

(3)

Core target layer, corresponding to people-oriented. “People-oriented” is the core goal of urban smart development. It contains citizens as the core service object, service, and value for the implementation of the people’s livelihood, with the people’s actual demand in the urban development as the basic goal of urban development and guides residents in seeking a superior work environment, life scenes, and community experience, providing a sustained and effective power.

2.1.2. Comprehensive Evaluation Index System

According to the PSF evaluation model, the three levels respectively correspond to the six level indexes of the comprehensive evaluation index system for urban intelligent development. Specific indexes are distributed as Figure 1.

The input support layer corresponds to the intelligent infrastructure elements. Infrastructure is the skeleton of the resource circulation exchange, with the aid of the grid system, Internet system, transport system, e-government system, medical system, and infrastructure that can greatly promote the urban traffic, meaning that resource and material flow has an efficient circulation and exchange. It is a city’s responsibility to implement the basic material safeguard of wisdom. The system application layer corresponds to sub-elements of wisdom, intelligence services management component factors, economic factors, and wisdom guarantee child elements. The application layer of the system not only needs to possess rich urban management experience and government administrative services, but also needs to have the corresponding economic development level to support and match the supporting social security, so as to guarantee the coordinated and fair development of urban wisdom. The core target layer corresponds to the elements of the intelligent crowd. The city of wisdom is ultimately required to serve the general population, and the original executor can be a part of it, so the people who have contributed to the city of wisdom and the benefits of the city should form the core of the target.

Wisdom city construction, management, and maintenance represents very complicated system engineering, so the evaluation of a city’s wisdom is extremely complex and involves the dimensions of the wisdom urban system characteristics. We need to achieve the goal of the city’s intelligence and development of the overall level of intelligence, and the country’s policy, inner law of cities, and the reality of urban acumen to lead the overall evaluation system. Based on the PSF evaluation model, the integrated evaluation system of urban intelligent development is divided into three levels by referring to the inclusion elements of each hierarchy. The first layer is the target layer, which represents the comprehensive evaluation index for the intelligent development of a city. The second layer is the criterion layer, also known as the primary index (B), respectively, for smart individual (B1), smart management (B2), smart services (B3), smart economy (B4), smart guarantee (B5), and smart infrastructure (B6). The comprehensive evaluation system of urban intelligent development is shown in

Table 1. Comprehensive evaluation index system for urban intelligent development.

Target Layer (A)	Primary Index (B)	Secondary Index (X)
Comprehensive evaluation index system for urban intelligent development	B1 Smart individual	X1 Information service industry practitioners
		X2 People’s life network level
	B2 Smart management	X3 Government online service level
		X4 Public resource trading platform
		X5 Social media engagement
	B3 Smart service	X6 Social welfare service level
		X7 Open data service levels
	B4 Smart economy	X8 Urban innovation and entrepreneurship level
		X9 Energy consumption level of economic output
		X10 Level of Internet industry development
	B5 Smart guarantee	X11 Development plan formulation
		X12 Information publicity and training
		X13 Performance appraisal
	B6 Smart infrastructure	X14 Basic network construction
		X15 Building and sharing of basic information resources
		X16 Application of urban Cloud Platform

. B1 explains the elements of wisdom in the core target layer and takes people as the basic unit of the wisdom city. B2, B3, B4, and B5 explain the four subsystems in the system application layer, and they can measure the level of intelligent application, economy, management, service, and guarantee of wisdom through the technical level and service level. B6 explains the basic elements of the urban basic material in the supporting layer.

3. Construction of Comprehensive Evaluation Model

3.1. Modeling Tools and Methods

3.1.1. Hybrid Neural Network Model

In the decision-making process, AHP fully embodies the high consistency with the human decision-making process, which is easy to use and widely applicable, but it is difficult to accurately describe the nonlinear relationship among variables as a linear model. The Artificial Neural Networks (ANNs) is a nonlinear model that has a strong tolerance for faults and the ability to generalize, a very good explanation for nonlinear models and parallel computational functions. In addition, it can realize self-learning, and with the enlargement of the training sample, the accuracy of the model will be higher and higher. But the single ANN cannot accurately describe the performance of each index and the weakness of the system structure. So, in order to give the model a good stability and accuracy, we use the AHP-ANNs hybrid method to build the Urban Intelligent Development Model.

In the hierarchical structure of the AHP, the first layer is the expected target of the research question, the middle layer is called the criterion layer (usually the primary index), and the lowest level is the scheme layer (usually the underlying index). There are $n$ elements in this structure, and their weights are ${\mathrm{W}}_i$, $i=1,2,3,\cdots ,n$ and ${\displaystyle \sum _{i=1}^n{\mathrm{W}}_i}=1$. We obtained a judgment matrix based on expert advice. Then, the consistency of hierarchical order ranking and judgment matrix is verified by Cholesky decomposition of the judgment matrix to obtain the maximum characteristic root ${\lambda }_{max}$ and feature vector ${\mathrm{W}}_i$.

To define the consistency index $CI=\frac{{\lambda }_{\max }-n}{n-1}$, if $CI=0$, the judgment matrix has a satisfactory consistency. The greater the value, the lower the consistency of the judgment matrix. The average random consistency index $CI$ is defined to represent the consistency test value of the judgment matrix of different order Numbers. With $CR=\frac{CI}{RI}$ as the consistency ratio, if $CR<0.10$, the judgment matrix is consistent. On the contrary, we think that the matrix is not consistent, and we need to adjust the judgment values $a_{ij}$. Finally, we need to check the total hierarchy sorting by calculating the level of all the elements for the target layer weight coefficient of hierarchy total sorting. If the consistency check of the total sequencing ratio $C{R}^{\prime }<0.10$, it passes this test. Conversely, the judgment matrix needs to be recalibrated.

The typical topology of the BP network is shown in Figure 2. The diagram contains an input layer consisting of $p$ neuron nodes, an implicit layer consisting of $m$ neuron nodes, and an output layer consisting of $q$ neuron nodes. $\mathrm{V}=\left[v_1,v_2,\cdots ,v_m\right]$ represents the weight of the connection input layer and the hidden layer neuron node, and $\mathrm{W}=\left[w_1,w_2,\cdots ,w_q\right]$ represents the weight of the neural node of the hidden layer and the output layer.

For the neuron nodes on the hidden layer:

$$ne{t}_j={\displaystyle \sum _{i=1}^p{v}_{ij}}\cdot x_i-{\theta }_j,j=1,2,\cdots ,m$$

$$o_j=f(ne{t}_j)$$

For the neuron nodes on the output layer:

$$ne{t}_k={\displaystyle \sum _{j=1}^m{w}_{jk}\cdot o_j}-{\theta }_k,k=1,2,\cdots ,q$$

$$y_k=f(ne{t}_k)$$

In this case, $f(•)$ is an activation function, usually a Sigmoid function.

$$f\left(x\right)=\frac{1}{1+e^{-x}}$$

The essence of the BP network learning process is to adjust the weights of the network through its target value $\mathit{t}$ and the predicted value $\mathit{y}$, and its learning process includes two stages: forward propagation and error retransmission. More specifically, in the forward propagation phase, the training samples are first passed into the network through the input layer and then transmitted to the output layer via the hidden layer. In the phase of error reverse propagation, the network adjusts the network error to the network to achieve the goal or reach the epoch, according to the error between the target value $\mathit{t}$ and the predicted value $\mathit{y}$.

Let’s say that $D_{train}$ is the training set ${\left\{\left(x_k,y_k\right)|x\in {ℝ}^p,y\in {ℝ}^q\right\}}_{k=1}^N$. The BP network uses the error function as the performance function to adjust the power value $\Delta \mathrm{W}$.

$$E=\frac{1}{2}{\Vert \mathit{t}-\mathit{y}\Vert }^2=\frac{1}{2}{\displaystyle \sum _{i=1}^n{\left(t_i-y_i\right)}^2}$$

$$\Delta \mathrm{W}=-\eta \frac{\partial E}{\partial \mathrm{W}}$$

For the input weight value $\mathrm{W}$ and the output weight $\mathrm{V}$, the updated formula is as follows:

$$\mathrm{W}\left(n\right)=\mathrm{W}\left(n-1\right)+\Delta \mathrm{W}\left(n\right)$$

$$\mathrm{V}\left(n\right)=\mathrm{V}\left(n-1\right)+\Delta \mathrm{V}\left(n\right)$$

Among them,

$$\{\begin{array}{c}\Delta w_{jk}=-\eta \frac{\partial E}{\partial w_{jk}}=\eta {\delta }_k^y{o}_j\\ \Delta v_{ij}=-\eta \frac{\partial E}{\partial v_{ij}}=\eta {\delta }_j^o{x}_i\end{array}$$

$$\{\begin{array}{l}{\delta }_k^y=-\frac{\partial E}{\partial y_k}{f}^{\prime }\left(ne{t}_k\right)=\left(t_k-y_k\right)y_k\left(1-y_k\right)\\ {\delta }_j^o=-\frac{\partial E}{\partial o_j}{f}^{\prime }\left(ne{t}_j\right)=\left({\displaystyle \sum _k{w}_{jk}{\delta }_k^y}\right)y_j\left(1-y_j\right)\end{array}$$

The above process is called the standard BP network. In order to further improve the performance of standard BP, we introduced a momentum factor, $\alpha$, so the weighting formula of the BP network was modified to:

$$\mathrm{W}\left(n\right)=\mathrm{W}\left(n-1\right)+\Delta \mathrm{W}\left(n\right)+\alpha \Delta \mathrm{W}\left(n-1\right)$$

$$\mathrm{V}\left(n\right)=\mathrm{V}\left(n-1\right)+\Delta \mathrm{V}\left(n\right)+\alpha \Delta \mathrm{V}\left(n-1\right)$$

The number of nodes of the hidden layer neurons can be determined by the empirical formula:

$$m=\sqrt{p+q}+a$$

$p$ represents the number of nodes of the input layer neuron, $q$ represents the number of nodes of the output layer neuron, and $a$ is the constant between $\left[0,10\right]$. In practice, the number of neuron nodes in the hidden layer needs to be determined by the Trial and Error method.

3.1.2. ELM Model

The extreme learning machine (ELM) model was first proposed by Huang et al. [34], from Nanyang Technological University in Singapore, and presented a strict theoretical proof in the literature [35]. Compared with the single-hidden layer feedforward neural network (SLFNs) based on gradient descent, ELM can randomly set the weight of the input and determine the size of the hidden layer weight with the least square method. ELM is an advanced machine learning technology, which is simple and easy to implement, and has great advantages in dealing with complex problems such as high-dimensional data sets and noise [36].

Set the training sample set $(x_i,y_i)$ to represent a collection of $N$ different samples. $x_i={[x_{i1},x_{i2},\dots ,x_{ip}]}^T\in {ℝ}^p$, $y_i={[y_{i1},y_{i2},\dots ,y_{iq}]}^T\in {ℝ}^q$. For the regression problem, $q=1$. For classification problems, $q$ represents categories or tags.

The ELM network has $Nh$ hidden nodes, and its mathematical model can be expressed as:

$$y_i={\displaystyle {\sum }_{j=1}^{Nh}{\beta }_{i}f(w_i\cdot x_j+b_i)}$$

$f\left(x\right)$ is the activation function, and it represents the following mapping relationship: $ℝ\to ℝ$.

The above equation can be expressed as the matrix form below:

$$\begin{array}{cc}\hfill Y& =H\mathsf{\beta }=[h_1,h_2,\dots ,h_{Nh}]\mathsf{\beta }\hfill \\ & =\left[\begin{array}{ccc}f(w_1\cdot x_1+b_1)& \dots & f(w_{Nh}\cdot x_1+b_{Nh})\\ \dots & f(w_i\cdot x_j+b_i)& \dots \\ f(w_1\cdot x_N+b_1)& \dots & f(w_{Nh}\cdot x_N+b_{Nh})\end{array}\right]\times \left[\begin{array}{c}{\beta }_1\\ \dots \\ {\beta }_N\end{array}\right]\hfill \end{array}$$

$w_i={[w_{i1},w_{i2},\dots ,w_{iq}]}^T$, $i=1,2,\dots ,Nh$ is the weight vector that connects the $i$ hidden neurons and inputs to the neuron; $\mathsf{\beta }={[{\beta }_{i1},{\beta }_{i2},\dots ,{\beta }_{iq}]}^T$ is the weight vector that connects to the $i$ hidden neurons and output neurons; and $H$ is the output matrix of the hidden layer.

Here, we take the Sigmoid as the activation function. The Sigmoid function is defined as:

$$f(x)=\frac{1}{1+e^{-x}}$$

We assume that the input weight $\mathrm{W}$ and the partial $b$ of the hidden neuron are known, so the only parameter of the linear model $Y=H\mathsf{\beta }$ can be expressed in this formula. The weight $\beta$ can be obtained by solving the least squares solution $\widehat{\beta }$ of the line

ar model.

$$\widehat{\beta }=\underset{\beta |w,b}{\min }\Vert \mathrm{H}\mathsf{\beta }-\mathrm{Y}\Vert$$

It can also be expressed as:

$$\widehat{\beta }={\mathrm{H}}^{†}\mathrm{Y}$$

Matrix ${\mathrm{H}}^{†}$ is the Moore-Penrose generalized inverse of matrix $\mathrm{H}$. It can be calculated by orthogonal projection, such as single value decomposition (SVD).

3.2. Synthesis and Processing of Comprehensive Evaluation Model

3.2.1. Data Normalization Processing

In the comprehensive evaluation index system of urban intelligent development, the provenance, type, and unit of raw data are different and the data cannot be compared directly. In order to follow-up data processing convenience and accelerate the convergence of data to ensure the subsequent simulation, we must therefore ensure the continuity and regularity of the data before the evaluation data normalization processing. Our paper adopts the Min-Max normalization method to deal with the cost and effect data.

3.2.2. The Determination of Index Weight

The target layer A in the comprehensive evaluation index system is the integrated evaluation index system of urban intelligent development, and the criterion layer B_i includes: smart infrastructure (B1), intelligent management (B2), intelligent service (B3), smart economy (B4), intelligent crowd (B5), and wisdom guarantee (B6). The scheme layer corresponds to the secondary index, which is recorded as X_i (i = 1, 2, …, 16). To find the weight of the criteria and the levels of the protocol, our paper builds a seven judgments matrix in the questionnaire, which are: Project target layer matrix, Smart infrastructure matrix, Intelligent management matrix, Intelligent service matrix, Intelligent economic matrix, Intelligent crowd matrix, and Wisdom assurance matrix. We used MATLAB R2017a to rank the hierarchy order and the total order of the above constructed judgment matrix, and the consistency test was conducted. Then, we adjusted the judgment value of the judgment matrix that failed to pass the consistency test, and the weight coefficient ${\mathrm{W}}_i$ and the consistency test of the final judgment matrix were as follows, as shown in

Table 2. The comprehensive evaluation index system weight of urban intelligent development.

Target Layer	Primary Index	Weight Coefficient	Consistency Test Results	Secondary Indicators	Weight Coefficient	Consistency Test Results
A1 Comprehensive evaluation index system for urban intelligent development	B1	0.1889	$\begin{array}{l}RC=0.077\\ {\lambda }_{\max }=6.48\end{array}$	X1	0.5	$\begin{array}{l}RC=0.004\\ {\lambda }_{\max }=2.0\end{array}$
				X2	0.5
	B2	0.2365		X3	0.33	$\begin{array}{l}RC=0.00\\ {\lambda }_{\max }=3.0\end{array}$
				X4	0.33
				X5	0.33
	B3	0.1156		X6	0.5	$\begin{array}{l}RC=0.00\\ {\lambda }_{\max }=2.0\end{array}$
				X7	0.5
	B4	0.0771		X8	0.25	$\begin{array}{l}RC=0.00\\ {\lambda }_{\max }=3.0\end{array}$
				X9	0.25
				X10	0.5
	B5	0.1534		X11	0.25	$\begin{array}{l}RC=0.00\\ {\lambda }_{\max }=3.0\end{array}$
				X12	0.5
				X13	0.25
	B6	0.2284		X14	0.2	$\begin{array}{l}RC=0.00\\ {\lambda }_{\max }=3.0\end{array}$
				X15	0.2
				X16	0.6

.

3.2.3. Determination of the Sample Target Value

Through the multi-factor comprehensive evaluation method, the evaluation formula of the urban intelligent development comprehensive evaluation index is expressed as:

$$\mathrm{Y}={\displaystyle \sum _{i=1}^{16}{\mathrm{X}}_i}\cdot {\mathrm{W}}_i$$

where $\mathrm{Y}$ is the comprehensive evaluation index, $X_i$ is the standardized value of the single index data, and $W_i$ is the weight corresponding to the index. ${\mathrm{W}}_i$ can be determined through AHP.

3.2.4. Comprehensive Evaluation Model

The comprehensive evaluation model of urban intelligent development was obtained through the BP and ELM model, as follows. The calculation process of the comprehensive evaluation model is shown in Figure 3.

$$\mathrm{Y}=f\left(\left(f\left(\mathrm{V}\cdot \mathit{x}-\mathit{\theta }\right)\right)\cdot \mathrm{W}-\mathit{\theta }\right)$$

$$Y=H\mathsf{\beta }=[h_1,h_2,\dots ,h_{Nh}]\mathsf{\beta }$$

4. Results

We looked up the China Statistical Yearbook in 2015 and the public information of China’s provincial statistics bureau [37]. The open data provided by the Tongheng smart city institute [38], Tsinghua university, is also one of the data sources of our paper. Since the time of the statistics is not synchronized, it is critical to ensure the reliability and integrity of the data, and given the sluggish nature of the intelligent cities to respond to national policies, we use the data from 2015 to analyze it. Here, we take 151 cities with different economic scales in China as the research objects, and collect the comprehensive evaluation index data of urban intelligent development. The sample set involves 151 records, with 16 inputs (i.e., 16 secondary indicators), which we classify as training sets and test sets. Among them, the training set contains 100 training samples, and the test set contains 51 test samples. In the process of model training, the training samples are required to be typical, uniform, and diverse, and we selected data from 20 cities as a training sample; the remaining 80 training samples are selected from the sample to be selected randomly from the sample. The 20 cities include those such as Beijing, Shanghai, and Hangzhou, which also include Ningbo and Wuxi, which exhibit relatively stable economic development. They also include Liupanshui, Yulin, and Anhua, and other less developed cities.

After normalization of the data, we calculated the weight coefficients of each index.

For AHP, the judgment matrix $A$ can be obtained by the expert survey method.

For BP, the activation function $f(•)$ is set to the Sigmoid function; The allowable error value (goal) is set to 1.00 × 10⁻⁶; The maximum iteration number (epoch) is set to 200; the momentum factor $\alpha$ is set to 0.95; and the learning rate $\eta$ is set to 0.7.

For ELM, the activation function $f(•)$ is also set to the Sigmoid function. In BP and ELM, the mean square error (MSE) of the test samples is used to measure the accuracy of the model.

For BP and ELM, the number of neurons in hidden layer nodes is obtained by the trial and error method. The BP model test error changes with the number of neuron nodes, as shown in

Table 3. BP model test error changes with the number of neurons in hidden layer nodes.

m	8	9	10	11	12	13	14	15
Testing Error	2.84 × 10−3	2.66 × 10−3	2.43 × 10−3	2.25 × 10−3	1.49 × 10−3	1.34 × 10−3	2.00 × 10−4	2.21 × 10−4
m	16	17	18	19	20	21	22	23
Testing Error	2.40 × 10−4	2.31 × 10−4	3.73 × 10−3	3.74 × 10−3	5.00 × 10−3	6.44 × 10−3	3.21 × 10−3	6.35 × 10−3

.

In the BP model, the number of neuron nodes in the hidden layer is set to 14. In this parameter setting, the training error and the test error are shown in Figure 4. The ELM model test error changes with the number of neuron nodes, as shown in

Table 4. ELM model test error changes with the number of neuron nodes.

m	8	9	10	11	12	13	14	15
Testing Error	3.46× 10−3	3.67 × 10−3	3.36 × 10−3	2.94 × 10−3	2.62 × 10−3	2.35 × 10−3	2.02 × 10−3	1.94 × 10−3
m	16	17	18	19	20	21	22	23
Testing Error	1.86 × 10−3	1.89 × 10−3	8.22 × 10−4	9.05 × 10−4	8.05 × 10−5	9.12 × 10−4	9.51 × 10−4	9.30 × 10−4

.

It can be seen that in the ELM model, the number of neuron nodes in the hidden layer is set to 20.

Some results of the comprehensive evaluation index of urban intelligent development using AHP, AHP-BP, and AHP-ELM are shown in

Table 5. The simulation results and classification results of the comprehensive evaluation index of the intelligent development of part cities.

	Cities	${\mathit{Y}}_1$	Classification	${\mathit{Y}}_2$	Classification	${\mathit{Y}}_3$	Classification
Training samples	Wuxi	0.8054	Excellent	0.7977	Good	0.8155	Excellent
	Shanghai	0.8113	Excellent	0.7952	Good	0.8013	Excellent
	Beijing	0.8195	Excellent	0.8021	Excellent	0.8165	Excellent
	Hangzhou	0.7992	Good	0.7824	Good	0.7788	Good
	Ningbo	0.7978	Good	0.7806	Good	0.7950	Good
	Shenzhen	0.7453	Good	0.7409	Good	0.7179	Good
	Zhuhai	0.6698	Good	0.6696	Good	0.6722	Good
	Foshan	0.6273	Good	0.6365	Good	0.6351	Good
	…	…	…	…	…	…	…
	Huainan	0.2863	Pass	0.2818	Pass	0.2948	Pass
Test samples	Yichun	0.3078	Pass	0.3121	Pass	0.3063	Pass
	Hulun Buir	0.3220	Pass	0.3225	Pass	0.3100	Pass
	Zunyi	0.3431	Pass	0.3490	Pass	0.3354	Pass
	Lianyungang	0.3556	Pass	0.3584	Pass	0.3471	Pass
	Xuzhou	0.3009	Pass	0.3000	Pass	0.2985	Pass
	Baoji	0.3078	Pass	0.3158	Pass	0.3084	Pass
	Anshan	0.3192	Pass	0.3176	Pass	0.3153	Pass
	Shijiazhuang	0.3036	Pass	0.2931	Pass	0.3148	Pass
	…	…	…	…	…	…	…
	Luohe	0.0533	Fail	0.0113	Fail	0.0439	Fail

, and the results of these three models correspond to $Y_1$, $Y_2$, and $Y_3$, respectively.

In order to make a better assessment of the level of urban intelligence, we have five evaluation criteria for the results of the sample values, which are “Excellent”, “Good”, “Fair”, “Pass”, and “Fail”. According to the above classification criteria, the classification results of the comprehensive evaluation model of intelligent development in three different cities are shown in Table 5. Figure 5 shows the distribution of the evaluation results obtained by using three methods.

AHP, AHP-BP, and AHP-ELM city wisdom of the development comprehensive evaluation index of the classification result is generally consistent, conforming to the law of development of the smart city. The comparison of the comprehensive evaluation index of urban intelligent development based on these three models is shown in Figure 6.

In the cities of Hangzhou, Ningbo, Qingdao, and Nanjing, there is a high energy efficiency, geographical environment, economic development, intelligent industry, and urban wisdom in the upper part of the country. In the AHP model, Hangzhou, Ningbo, Shenzhen, Zhuhai, Foshan, Xiamen, Guangzhou, Qingdao, and Nanjing are rated as “good”, and the AHP-BP model and the AHP-ELM model are consistent with the AHP model.

Cities such as Luzhou, Qinzhou, and Haidong are restricted by geographical environment, low economic development level, imbalance of urban industrial structure, high brain drain rate, low urban innovation vitality, and corresponding smart city infrastructure construction level backward. These cities’ intellectual development degree is in the lower reaches of China. In the AHP, AHP-BP, and AHP-ELM model, all seven cities were rated as “Fail”, and the results of these three were consistent.

In fact, Beijing and Shanghai are the first of the nation’s smart cities. On the one hand, they have a huge economic scale and a high concentration of resources, but on the other hand, with good infrastructure to build, and better management and services, these cities’ intellectual development is at the forefront of China. In the results of the AHP and AHP-ELM model, Beijing, Shanghai, and Wuxi are classified as “excellent” grades, which are consistent with the reality. However, in the results of the AHP-BP model, Wuxi and Shanghai were divided into “good” grades, which did not tally with their actual development conditions.

From the above analysis, compared with the AHP-BP model, the AHP-ELM model is more accurate for the comprehensive evaluation of urban acumen. As can be seen from Table 3 and Table 4, the test error of AHP-BP is 2.00 × 10⁻⁴ and of AHP-ELM is 8.05 × 10⁻⁵. Obviously, the accuracy of the AHP-ELM model is better than the AHP-BP model. Besides,

Table 6. AHP-BP and AHP-ELM computation time costs.

	AHP-BP	AHP-ELM
Computation Cost (s)	14.11 s	0.42 s

shows that the AHP-ELM model takes a very short time.

Considering the factors of model precision and calculation costs, we believe that the performance of the AHP-ELM model is far superior to that of the AHP-BP model. So, this hybrid evaluation model has improved and focused on the information about urban informationization, without using the information technology infrastructure as a critical point, and thinking about the hardware and other dimensions that smart cities need to develop. In addition, the application of the PSF model also improves the subjectivity of the evaluation index. Moreover, to a certain extent, it eases the wisdom city facing the broad indicators on the evaluation of the problem. In the sample of China’s cities, we discovered that the AHP-ELM model could be effective in breaking the bottleneck in intelligent city evaluation.

5. Further Research

Although our research can provide guidance for the design of comprehensive evaluation indexes of urban intelligent development to some extent, we can construct a reference model for the evaluation of urban intelligent development, but for the shortcomings in our research, we can also consider the following perspectives in future work.

In the future, we can further enrich our evaluation index system, which may include intelligent humanity (education is also included in this category), technological innovation capability, and green development capacity. This will give a more complete explanation of the level of humanistic innovation and green development in the construction of smart cities. In addition, more types of evaluation models can be used for comprehensive comparison. The practicability of the model can also be verified by time series analysis. In the current smart city evaluation index system, some indexes are difficult to quantify, and the fuzzy theory can be considered to deal with such data. Our overall assessment of the development of urban intelligence is a static assessment, and given that the development of the city is made by the dynamic evolution of the economic, environmental, social, and technological subsystems, we can use the method of dynamic assessment to capture the potential dynamics of the development of the city.

6. Conclusions

From the perspective of the evolution logic of the urban system, a smart city is the only solution to the problems and contradictions that have become increasingly intensified in the process of urban development. Based on the PSF evaluation model, we constructed a comprehensive evaluation index system for urban intelligent development, and used the analytic hierarchy process and neural network as modeling tools to construct AHP-BP and AHP-ELM. We then tested the performance of the model in 151 cities in China. We have established a comprehensive evaluation index system for urban intelligent development. Based on the analysis of the overall assessment of the development of intelligent cities in China, we have built a model of the integrated assessment of urban intelligence based on the theory of the PSF model. In addition, we have constructed a comprehensive evaluation model of urban intelligent development. Using the AHP model synthetic evaluation index as the benchmark value, the integrated evaluation model of urban intelligent development is constructed by using the neural network model.

Experimental results show that the test error of the AHP-ELM model is less than that of AHP-BP in model precision. In addition, in terms of computational overhead, the AHP-ELM model is much less time-consuming than the AHP-BP model.