An Integrated Indicator System and Evaluation Model for Regional Sustainable Development

Abstract

Regional sustainable development has become a worldwide issue in recent years, but there is no single and universally agreed method of choosing indicators for sustainable development assessment. The subjective selection of indicators will affect the results of assessment. Each evaluation method has its own advantages and disadvantages, and the methods used to determine indicator weight also differ. Regional sustainable development is a complex system, which is difficult to evaluate objectively and scientifically using a single method. Therefore, a new integrated indicator system and evaluation model is constructed here to more accurately reflect regional sustainable development level. The indicator system and evaluation model were constructed using a case study of 17 cities in Shandong Province, China. The indicator system includes 4 subsystems, i.e., economy, society, resource, and environment. These indicators were selected through correlation analysis and discrimination analysis. A back propagation neural network was applied to evaluate the respective scores of the 4 subsystems. The comprehensive score for regional sustainable development was evaluated using the analytic hierarchy process with entropy correction. The results show that sustainable development levels in these 17 cities show a gradually decreasing trend from east to west and from coast to inland. Cities with an underdeveloped economy usually display poor levels of social development and serious environmental pollution. Through the improvement of indicator screening, evaluation model, and result correction, the error caused by a single evaluation method can be reduced significantly. This new methodology for indicator selection and comprehensive evaluation provides a new perspective for the assessment of regional sustainable development.

1. Introduction

Excessive resource consumption, high population levels, severe environmental pollution and rapid climate change represent significant challenges for achieving sustainable development around the world. Breaking the globe into regions provides an approach to tackling worldwide sustainable development. A quantified indicator system and evaluation model can be used to reflect the current level of regional sustainable development; this baseline can then inform the development of corresponding policies and plans.

There has been a good deal of research in the field of regional sustainable development. Sala et al. [1] present a novel methodological framework for sustainability evaluation models and indicators by addressing critical decision-making elements to recognize the ontological, epistemological, and methodological foundations of sustainability. Arushanyan et al. [2] present a new sustainability assessment framework to assess the environmental and social risks and opportunities in future scenarios. Their qualitative assessment of environmental and social aspects uses a consumption perspective. Popovic et al. [3] developed a clear definition of quantitative indicators that can be used to perform social sustainability assessment. Research has also explored the application of multi-methodology strategies to define a set of indicators for quantitative assessment of social sustainability in supply chains based on uncertainty analysis of different sustainability assessment methods. Reznichenko et al. [4] propose that modern institutional conversions confirm the relevance of the research on problems of sustainable development of socio-economic systems and the identification of effective management tools at the level of regions. Some of the most important issues in ensuring sustainable regional development are the state of the social and production infrastructure, the provision of the region with food, fuel, and energy resources. For example, high dependence on outdated and inefficient manufacturing processes is likely to impact on resources and environmental aspects of the region. Similarly, the choice of renewable or non-renewable energy sources is likely to impact on environmental quality. Mally [5] presents the findings of a study that used a selection of 32 economic, social, and environmental indicators to evaluate the extent of achieving these objectives in Slovenian statistical regions from 2010 to 2014.

China has been undergoing rapid economic development over the past few decades, but this progress has also brought serious environmental pollution. In order to achieve sustainable development, the Chinese government has issued a series of policies and plans [6,7,8]. However, a key issue in measuring sustainable development is how to establish an indicator system and evaluation method. Liu [9] presented an effective framework of general sustainability indicators for renewable energy systems. Naganathan and Chong [10] addressed issues of quantification and policy intent through the proposed State Sustainable Transportation Performances matrices, and ranked sustainable transportation performance based on the indicators of the USA. Wu et al. [11] proposed a new hybrid evaluation method based on an analytical hierarchy process (AHP)-entropy weight and the cloud model to evaluate community sustainability. This method makes use of the superiority of the cloud model to transform qualitative remarks into quantitative representations to reflect fuzziness and randomness. Davor et al. [12] presented an empirical study to assess the sustainability performance of European countries using differential multi-criteria analysis. Their Preference Ranking Organization Method for Enrichment Evaluations was applied on 38 headline and operational sustainable development indicators defined under the EU Sustainable Development Strategy.

Sustainable development has been a basic principle of national development in many countries, at both the regional level and city level, but existing problems and the level of sustainable development vary across different regions [13,14]. A quantitative evaluation of the sustainable development level of different regions is useful in understanding constraints and proposing corresponding counter measures to enhance sustainability. Phillis et al. [15] selected 46 indicators across 7 aspects to evaluate the sustainable development level of 106 cities in different countries using the fuzzy evaluation method. The results can guide decision makers in allocating their available resources to one or more indicators to obtain the largest improvement of sustainability. Elgadi et al. [16] constructed an indicator system with three levels and four categories to evaluate the sustainable development of Tripoli through primary index screening. Using an indicator system of three fields, Widomski et al. [17] compared the indicator values of Lublin, Poland with other EU countries, and evaluated its sustainable development level. Tsai [18] selected a set of indicators divided into six dimensions to analyze the sustainable development trend of Taiwan over several years using the Pressure-State-Response (PSR) model. Davor et al. [19] investigated the feasibility of modeling municipal waste generation for countries at different levels of development using artificial neural networks (ANN) and selected generic indicators of sustainability. Based on a comparison of actual municipal waste generation data with predictions given by the model, it showed that ANN can be applied to modeling and forecasting municipal waste generation on a national scale. Ugwu and Haupt [20] proposed a comprehensive model to assess the sustainability of infrastructure projects from the aspects of economy, environment, society, resource utilization, health and safety, and project management. Marynych [21] used a methodology based on the vector autoregressive and Johansen VEC approach procedures. The results of the estimation of the aggregate index of sustainable development showed a significant impact and variability in the economic component, which had further influence on social and environmental indicators. These studies demonstrate that the international evaluation of sustainable development started years ago, but there is no unified determination method for indicator systems, and the conclusions using different systems are often quite different.

Existing indicator systems for assessing sustainable development can be divided into two kinds: single and comprehensive evaluation indicator systems. The former adopts a hybrid indicator system to summarize progress on regional sustainable development. Examples are genuine saving [22], energy analysis [23,24], ecological footprint [25,26], the human development index [27,28], and the social progress index [29]. These indicator systems work well in respect of specific criteria, but less well as a general measure of sustainability.

The latter, comprehensive indicator systems, analyse regional sustainable development levels with multi-hierarchy indicators, such as the PSR model, three component model [30], or an indicator system based on AHP. They can reflect sustainable development levels widely and clearly, but data acquisition and results calculation become difficult once the indicator system is complex. With respect to the evaluation methods of sustainable development level, AHP [31], principal component analysis (PCA) [32], data envelopment analysis (DEA) [33], technique for order preference by similarity to ideal solution (TOPSIS) [34], and ANN [35] are often applied. The AHP method is used most frequently, and is usually combined with the entropy method and fuzzy comprehensive evaluation because of its strong subjectivity. The weight determination of PCA is objective, but the correlation among indicators should be significant. The DEA model is also objective, but it can only evaluate relative efficiency. TOPSIS has simple principles and fast calculations, but can only evaluate the relative strengths and weaknesses of objects, and the weight component may suffer from subjectivity. ANN can realize the fitting of arbitrary nonlinear functions, which is consistent with the characteristics of the complex nonlinear system of regional sustainable development. At present, the ANN algorithm is complex. If the sample cannot guarantee accuracy, the findings may be incorrect.

In summary, two existing problems need to be improved upon in the current study. Firstly, there is no single and universally agreed upon method for choosing indicators. The subjective selection of indicators will affect the results. Secondly, each evaluation method has its own advantages and disadvantages, and the methods used to determine indicator weight also differ. It should be ensured that the evaluation system can evaluate the regional sustainable development level accurately and comprehensively.

In this research, a comprehensive selection of indicators was the first step. These indicators were then screened by means of discrimination analysis and correlation analysis to reduce the subjectivity of indicator selection. Spurious relationships among sustainable development indicators were found and improved using partial correlation analysis. This method ensures the accuracy of the selected indicators. ANN and AHP with entropy correction were applied to evaluate and rank the levels of regional sustainable development. This improvement of indicator screening, evaluation model, and result correction can reduce the error caused by single evaluation method significantly. This new indicator system and evaluation model can reflect the regional level of sustainable development accurately, which provides theoretical guidance for increasing the capability of sustainable development in different regions.

2. Research Method

2.1. Area of Study

Shandong Province is one of the more economically developed provinces in China, and covers an area of 157.1 thousand km². Its GDP amounted to 7646.97 billion RMB in 2018, which accounts for 8.5% of national GDP [36]. The population of Shandong Province is 0.1 billion, ranking it second place in Chinese provinces. The social and economic situation of Shandong Province is similar to China as a whole, with more economically developed regions on the eastern coastline and underdeveloped regions in the western inland areas. The level of development of economy, society, resources, and environment in the 17 cities of Shandong Province is variable. Thus, Shandong Province is a typical representative of the whole country. Figure 1 shows the location of Shandong Province in China. Figure 2 shows the geographical locations of the 17 cities in Shandong Province.

In order to construct a more objective and scientific indicator system and evaluation model for regional sustainable development, discrimination analysis, Pearson correlation analysis, partial correlation analysis, back propagation (BP) ANN, and AHP with entropy correction were applied.

2.2. Construction of Indicator System

To construct the indicator system for regional sustainable development, it is necessary to follow principles of hierarchy, simplicity, comprehensiveness, and operability [37]. Referring to the viewpoints of the China Sustainable Development Strategy Report 2016 [38], Widomski et al. [17] and Phillis et al. [15], and the framework of the Drive-State-Response (DSR) model proposed by UNCSD [39], the indicator system is divided into four layers, i.e., object layer, system layer, feature layer, and indicator layer. The sustainable development system is divided into four subsystems, i.e., the economy, society, resources, and environment. Figure 3 shows the layout of the indicator system for regional sustainable development. The indicators were screened using discrimination analysis and correlation analysis after the initial comprehensive selection of indicators.

2.3. Discrimination Analysis

The discrimination of an evaluation indicator refers to its ability to distinguish the feature differences of the objects evaluated. If an indicator has similar values for all evaluated regions, it means the discrimination of this indicator is too weak to recognize the difference of sustainable development level for these regions. In practice, the coefficient of variation is usually used to describe the discrimination of the indicators [40], see Equation (1).

$$V_i=\frac{S_i}{\overline{X}}$$

(1)

where $\overline{X}=\frac{1}{n}{\displaystyle \sum _{i=1}^n{X}_i}$ is the average value, and $Si=\sqrt{\frac{1}{n-1}{{\displaystyle \sum \left(X_i-\overline{X}\right)}}^2}$ is the standard deviation. The indicator discrimination was analyzed using SPSS 19.0 software to determine the descriptive value, and assess the discrimination of indicators. The smaller the coefficient of variation V_i, the weaker the indicator discrimination will be. By comparing V_i and the critical value, smaller V_i values were deleted to ensure the simplicity and comprehensiveness of the selected indicators.

2.4. Correlation Analysis

2.4.1. Pearson Correlation Analysis

There were some correlations among the indicators in the system, and the repeated information reflected by them would impact the results of the regional sustainable development evaluation. The correlation coefficient between two indicators was calculated by Pearson correlation analysis [41]. One of the indicators was deleted if the correlation coefficient between two indicators was high, so as to reduce the impact of repeated information reflected by the indicators on the evaluation results. This was done in three steps [42]:

Firstly, the standardization of the original value was calculated with Equation (2):

$$Z_i=\frac{X_i-\overline{X}}{S_i}$$

(2)

where Z_i is the standardized value, X_i is original value of an indicator, and S_i is the standard deviation.

Secondly, the correlation coefficient R_ij between two indicators was calculated with Equation (3):

$$R_{ij}=\frac{{\displaystyle \sum _{k=1}^n\left(Z_{ki}-{\overline{Z}}_i\right)}\left(Z_{kj}-{\overline{Z}}_j\right)}{\sqrt{{\displaystyle \sum _{k=1}^n{\left(Z_{ki}-{\overline{Z}}_i\right)}^2{\left(Z_{kj}-{\overline{Z}}_j\right)}^2}}}$$

(3)

Thirdly, a critical value A was set. If R_ij > A, the indicator or whichever has less significance was deleted. Otherwise, both indicators were retained.

2.4.2. Partial Correlation Analysis

Partial correlation analysis is a method where the impact of other elements is regarded as constant when researching the correlation between two factors in a multi-element system [43]. In this way, the partial correlation between two evaluation indicators can be identified, which can enhance the accuracy and reliability of the evaluation system. The partial correlation coefficient is calculated by Equation (4).

$$R_{ij(k)}=\frac{R_{ij}-R_{ik}{R}_{jk}}{\sqrt{1-R_{ik}^2}\sqrt{1-R_{jk}^2}}$$

(4)

Due to the complicated relations among various factors in the regional sustainable development system, it was difficult to select control variables. Some policy- or management-related factors cannot be quantified, which leads to partial correlation analysis failure due to the lack of data availability. Therefore, an expert judgment method was adopted and the correlation between two indicators was classified on a four-point differential as one of: no correlation, weak, strong, or very strong. One of the two indicators in pairs with a strong or very strong correlation was deleted.

2.5. Back Propagation Artificial Neural Network

BP ANN is a multi-layer feed-forward neural network; it can approximate complex nonlinear functions with any precision [44]. It is widely used due to its simple structure, rich algorithm, and strong nonlinear mapping capability. The learning algorithm of BP ANN is the error back propagation algorithm. Therefore, as long as the parameters are fully adjusted and the samples are accurate and reliable, errors generated by the neural network can be significantly reduced. The indicator system and evaluation model can accurately reflect the sustainable development level of each region. MATLAB software was applied to establish the BP ANN model of regional sustainable development evaluation and calculate the scores of the four subsystems for sustainable development in different sample cities.

(1) Data normalization

$$X_i^{\prime }=\frac{X_i-X_{min}}{X_{max}-X_{min}}$$

(5)

In Equation (5), X_i is the indicator value of one evaluated object. X_i′ is the normalized value. X_min is the minimum indicator value. X_max is the maximum indicator value. For the negative indicators, the final value is 1 minus the normalized value.

(2) Building training samples

The calculation of BP ANN requires predicted output, which is unknown for evaluation. Based on the linear interpolation of sample data between min and max, influence levels are set to generate training samples [45,46,47,48,49]. In this research, 500 training samples were established. Referring to Kennedy et al. [45] and Sun et al. [46], and combining with the expert judgment method, sustainable development capacity was classified into 5 levels: (0, 1] means very low level, (1, 2] means low level, (2, 3] means moderate level, (3, 4] means high level, (4, 5] means very high level.

(3) The establishment of the back propagation artificial neural network

a. The quantification of hidden layers

The three-layer neural network applied in this research can approach continuous function precisely [49]. Increasing the number of hidden layers can reduce errors, but it will also increase the training time.

b. The quantification of hidden layer nodes

Fewer hidden layer nodes will decrease the ability to find the trend of samples in a network and result in too many minimum points. By contrast, more hidden layer nodes can increase the training time. The general way to determine the optimal number of hidden layer nodes is cut and trial method. Varying the number using a cut and trial method can be useful to optimize the number of hidden layer nodes [50]. The estimated value was used as the initial value of the cut and trial method, which was obtained by Equations (6)–(9) respectively. The number of hidden layer nodes was increased gradually. After comparing the prediction performance of the network, the number of nodes with the best performance was selected as the number of hidden layer nodes.

$$s=lo{g}_{2}m$$

(6)

$$s=\sqrt{mn}$$

(7)

$$s=\sqrt{m+n}+a$$

(8)

$$s=\sqrt{0.43mn+0.12n^2+2.54m+0.77n+0.35}+0.51$$

(9)

where S is the number of hidden layer nodes, m is the number of input nodes, n is the number of output nodes, and a is a constant between 1 and 10.

c. The selection of transfer function

For nonlinear mappings, the sigmoid transfer function [51] and linear transfer function are usually applied for the hidden layer and output layer. The output values of the entire network can be arbitrary. In this research, logsig or tansig was selected as the hidden layer transfer function, purelin was selected as the output layer transfer function, and traingdx or trainlm was selected as the training function.

d. The determination of learning parameters

The range of learning rates was set in the range from 0.01 to 0.8. A higher rate may result in network instability, while lower rates may increase training time. Generally, a lower rate is used in order to make the network converge easily.

e. The selection of training function

The improved training algorithm is usually adopted for BP ANN. Furthermore, the additional momentum factor may cause the network to slip through the local minimum point, which can speed up the convergence rate. Adaptive learning efficiency can adjust learning efficiency automatically and enhance the network’s stability. The Levenberg–Marquardt (LM) algorithm has a quick convergence rate, which is suitable for small and medium sized networks. Based on these analyses, the gradient descent method, traingdx with adaptive learning efficiency and additional momentum, and trainlm of LM algorithm were adopted in this research.

2.6. Analytic Hierarchy Process with Entropy Correction

AHP with entropy correction was selected to calculate overall scores of regional sustainable development levels. The main steps included establishing a multi-hierarchy model, structuring a judgment matrix, level ranking, and consistency checking [52]. The weight coefficients were modified by entropy correction in order to ensure the consistency of the judgment matrix. The main steps in entropy correction were as follows:

(1) Calculating the output entropy e_j of indicator f_j.

The judgment matrix ${(a_{ij})}_{m\times n}$ was normalized by formula ${\overline{a}}_{ij}/{\displaystyle \sum _{j=1}^n{a}_{ij}}$ in order to get the standard matrix $\overline{A}={({\overline{a}}_{ij})}_{n\times m}$. The output entropy e_j of indicator f_j was calculated with Equation (10).

$$e_j=-\frac{1}{lnn}{\displaystyle \sum _{j=1}^n{a}_{ij}ln{\overline{a}}_{ij}}$$

(10)

(2) Calculating the deflection degree d_j of indicator f_j with Equation (11).

$$d_j=1-e_j$$

(11)

(3) Calculating the information weight μ_j of indicator f_j with Equation (12).

$${\mu }_j=\frac{d_j}{{\displaystyle \sum _{j=1}^n{d}_j}}$$

(12)

(4) Calculating the modified indicator weight coefficient λ_j with Equation (13).

$${\lambda }_j=\frac{{\mu }_j{\omega }_j}{{\displaystyle \sum _{j=1}^n{\mu }_j{\omega }_j}}$$

(13)

Figure 4 is the technical roadmap, which shows the concept and methods of constructing the indicator system and evaluation model for regional sustainable development.

3. Results

3.1. Initial Selection of Indicators

The input data of the 17 sample cities in Shandong Province were retrieved from the Shandong Provincial Statistical Yearbook 2016 [53], and the statistical yearbooks and statistical bulletins of the 17 cities, respectively. Referring to the views of Phillis et al. [15], Popovic et al. [3], and Tran [39] there were 58 indicators in the indicator system, namely: (1) 13 indicators in 5 categories for the Economic subsystem, (2) 26 indicators in 5 categories for the Social subsystem, (3) 7 indicators in 3 categories for the Resource subsystem, and (4) 12 indicators in 3 categories for the Environmental subsystem, see

Table A1. Original indicator system for evaluation of regional sustainable development.

Object Layer	System Layer	Characteristic Layer	Indicator Layer	No.
Regional sustainable development	Economic development	Economic scale	GDP per capita (RMB)	d1
			Proportion of public finance budget in GDP (%)	d2
			Total fixed assets investment per capita (RMB)	d3
		Economic structure	Proportion of the secondary industry in GDP (%)	d4
			Proportion of the tertiary industry in GDP (%)	d5
		Economic development rate	Growth rate of GDP (%)	d6
			Growth rate of the output value of the secondary industry (%)	d7
			Growth rate of the output value of the tertiary industry (%)	d8
		Export-oriented economy	Total export-import volume per capita (millions US$)	d9
		Economic benefit	Elasticity coefficient of energy consumption (%)	d10
			Labour productivity per capita (thousand RMB)	d11
			Industrial added value per capita (thousand RMB)	d12
			Proportion of high and new technology industry output value in total designated size enterprises (%)	d13
	Social progress	Population development	Population density (person/km2)	d14
			Population growth rate (‰)	d15
			Proportion of people over 65 years old (%)	d16
			Urbanization rate (%)	d17
			Number of students in ordinary universities for every thousand persons	d18
			Number of books in public libraries per capita	d19
		Living standard	Disposable income of urban residents per capita (thousand RMB)	d20
			Engel coefficient of urban residents (%)	d21
			Disposable income of rural residents per capita (thousand RMB)	d22
			Engel coefficient of rural residents (%)	d23
			Total volume of retail sales per capita (million RMB)	d24
			Number of private vehicles per million persons	d25
			Living area per capita in urban (m2)	d26
		Infrastructure construction	Water supply penetration (%)	d27
			Nature gas supply penetration (%)	d28
			Number of urban public transport vehicles for every million persons	d29
			Area of urban road per capita (m2)	d30
			Density of the drainage pipes in built-up areas (km/km2)	d31
		Science and technology	Proportion of R&D cost in GDP (%)	d32
			Total number of R&D (person)	d33
		Social stability and security	Registered urban unemployment rate (%)	d34
			Number of hospital beds per thousand persons	d35
			Number of doctors per thousand persons	d36
			Coverage rate of endowment insurance (%)	d37
			Coverage rate of medical insurance (%)	d38
			Retirement pay per capita (thousand RMB)	d39
	Resources condition	Resources conservation	Water resources per capita (m3)	d40
			Area of cultivated land per capita (m2)	d41
			Forest coverage rate (%)	d42
		Resource consumption	Electricity consumption per capita (MWh)	d43
			Water consumption per captia (m3)	d44
		Resource utilization efficiency	Energy yield (thousand RMB/tce)	d45
			Water resource yield (RMB/m3)	d46
	Ecological environment	Environmental state	Ammonia-nitrogen emissions per unit GDP (t/billion RMB)	d47
			COD emissions per unit GDP (t/billion RMB)	d48
			SO2 emissions per unit GDP (t/billion RMB)	d49
			NOx emissions per unit GDP (t/billion RMB)	d50
			Soot emissions per unit GDP (t/billion RMB)	d51
			Industrial solid waste emissions per unit GDP (t/billion RMB)	d52
		Pollution control	Treatment rate of domestic sewage (%)	d53
			Innoxious treatment rate of household garbage (%)	d54
			Comprehensive utilization rate of industrial solid waste (%)	d55
		Natural ecological protection	Area of public park per capita (m2)	d56
			Coverage rate of green area in built-up areas (%)	d57
			Proportion of natural reserve area to total land area (%)	d58

in the Appendix A.

3.2. Confirmation of the Indicator System

Due to data unavailability, d10, d16, and d58 were deleted. Indicators d14 and d15 were the moderate indicators, which had different optimum populations for different regions. In order to ensure the accuracy of the results, d14 and d15 were deleted [54]. The 53 remaining indicators were then tested using discrimination analysis, Pearson correlation analysis, and partial correlation analysis.

3.2.1. Discrimination Analysis

Table 1. Coefficient of variation of each indicator after initial selection.

Indicator	Coefficient of Variation	Indicator	Coefficient of Variation	Indicator	Coefficient of Variation
d1	0.48	d23	0.09	d41	0.29
d2	0.16	d24	0.37	d42	0.27
d3	0.54	d25	0.32	d43	0.88
d4	0.11	d26	0.10	d44	0.44
d5	0.14	d27	0.00	d45	0.34
d6	0.10	d28	0.01	d46	0.72
d7	0.14	d29	0.68	d47	0.43
d8	0.12	d30	0.16	d48	0.49
d9	1.25	d31	0.30	d49	0.73
d11	0.48	d32	0.24	d50	0.65
d12	0.67	d33	0.77	d51	1.77
d13	0.24	d34	0.20	d52	3.65
d17	0.13	d35	0.15	d53	0.01
d18	0.85	d36	0.21	d54	0.00
d19	0.47	d37	0.07	d55	0.08
d20	0.21	d38	0.04	d56	0.26
d21	0.10	d39	0.13	d57	0.05
d22	0.15	d40	0.36

Bold means value less than 0.12.

shows the coefficient of variation of each indicator calculated by SPSS. According to the data distribution and the importance of each indicator in sustainable development of Shandong Province, 0.12 was selected as the critical value of the coefficient of variation, referring to Arushanyan et al. [2], Chen et al. [13], and Kılkış [23], The indicators with variation coefficients less than 0.12 were d4, d6, d21, d23, d26, d27, d28, d37, d38, d53, d54, d55, and d57. Indicator d6 was retained because it was one of the most appropriate indicators to reflect economic development. Forty one indicators were retained after discrimination analysis.

3.2.2. Pearson Correlation Analysis

An absolute value of correlation coefficient less than 0.3 indicates no linear correlation, 0.3–0.5 indicates low correlation, 0.5–0.8 indicates moderate correlation, and above 0.8 indicates significant correlation [55,56]. If the correlation coefficient ≥0.8, it means that the correlation is high for the two indicators and one of these two indicators should be deleted. In this way, d7 and d11 were deleted in the Economic subsystem. All indicators were retained in the Resources subsystem. In the Social subsystem, d24, d25, and d35 had high correlation, but these indicators were retained because they were the crucial indicators in relation to the total emissions control target in The 13th Five-Year Plan for National Eco-Environmental Conservation [7].

3.2.3. Partial Correlation Analysis

Analyzing the partial correlation of d1 and d3 as examples, d12 was regarded as a control variable. When the control variables were not controlled, the correlation coefficient between d1 and d3 was 0.961, and the significance level was 0.000, which is highly correlated and significant. The correlation coefficient dropped down to 0.587 when considering d12 as a control variable; as a result, the significance level was 0.017. See

Table 2. Correlation coefficient between d1 and d3 with or without control variable d12.

Control Variable			d1	d3	d12
None	d1	Correlation coefficient	1.000	0.961	0.942
		Significance (both sides)	0	0	0
		df	0	15	15
	d3	Correlation coefficient	0.961	1.000	0.966
		Significance (both sides)	0	0	0
		df	15	0	15
	d12	Correlation coefficient	0.942	0.966	1.000
		Significance (both sides)	0	0	0
		df	15	15	0
d12	d1	Correlation coefficient	1.000	0.587
		Significance (both sides)	0	0.017
		df	0	14
	d3	Correlation coefficient	0.587	1.000
		Significance (both sides)	0.017	0
		df	14	0

.

It can be seen that d12 was significantly correlated with d1 and d3 at the same time. According to the correlation logic, d1 and d3 should also be significantly correlated, but partial correlation analysis showed that when d12 was regarded as a control variable, the correlation coefficient between d1 and d3 decreased significantly. Indicators d1 and d3 were moderately correlated. Therefore, d1 and d3 can be considered to reflect the characteristics of different aspects within the economic subsystem. Both d1 and d3 were retained in the final indicator system.

According to partial correlation analysis, there may be a spurious relationship between two indicators with a high correlation coefficient. After correlation analysis, expert judgement was applied to delete indicators with high correlation. Indicators d7, d11, d24, d25, and d35 were deleted in this step.

Following indicator selection, the final indicator system with 36 indicators is shown in

Table A2. The indicator system for evaluation of regional sustainable development.

Object Layer	System Layer	Characteristic Layer	Indicator Layer	No.	Value of Indicators
					Jinan	Qingdao	Zibo	Zaozhuang	Dongying	Yantai	Weifang	Jining	Tai’an
Regional sustainable development	Economic development	Economic scale	GDP per capita (thousand RMB)	D1	85.9	102.5	89.2	52.7	163.9	92	55.8	48.5	56.5
			Proportion of public finance budget in GDP (%)	D2	10.07	10.82	7.7	7.35	6.38	8.42	9.37	9.19	6.5
			Total fixed assets investment per capita (thousand RMB)	D3	49.1	72.1	58.8	41.9	146.2	66.5	48.7	34.8	46.7
		Economic structure	Proportion of the tertiary industry in GDP (%)	D4	57.2	52.8	42.5	39.7	31.9	41.6	43	41.4	45.2
		Economic development rate	Growth rate of GDP (%)	D5	8.06	8.06	7.1	7.07	6.86	8.37	8.31	8.4	8.05
			Growth rate of the output value of the tertiary industry (%)	D6	8.89	9.42	7.8	8.3	7.86	9.84	9.84	10.67	9.89
		Export-oriented economy	Total export-import volume per capita (million US$)	D7	1.39	7.72	1.64	0.41	6.12	7.04	2.04	0.66	0.41
		Economic benefit	Industrial added value per capita (thousand RMB)	D8	25.91	39.03	42.24	24.78	103.84	42.74	23.51	20.16	22.2
			Proportion of high and new technology industry output value in total designated size enterprises (%)	D9	42.63	41	31.83	19.83	35.02	41.11	31.85	28.87	26.85
	Social progress	Population development	Urbanization rate (%)	D10	67.96	69.99	67.26	53.46	65.52	60.35	55.8	52.75	57.04
			Number of students in ordinary universities for every thousand persons	D11	75.18	35.43	22.19	7.84	14.23	26.03	16.92	12.53	19.46
			Number of books in public libraries per capita	D12	0.56	0.66	0.53	0.36	0.61	0.82	0.42	0.23	0.28
		Living standard	Disposable income of urban residents per capita (thousand RMB)	D13	39.89	40.37	33.79	25.79	38.74	35.91	31.06	27.89	28.13
			Disposable income of rural residents per capita (thousand RMB)	D14	14.23	16.73	14.53	12.04	13.89	15.54	14.89	12.57	13.32
		Infrastructure construction	Number of urban public transport vehicles for every million persons	D15	94	96	48	39	37	35	19	24	29
			Area of urban road per capita (m2)	D16	27.58	23.47	23.40	25.50	34.76	21.32	28.55	31.85	25.85
			Density of the drainage pipes in built-up areas (km/km2)	D17	6.70	12.35	10.09	8.34	9.45	11.85	12.40	9.18	6.87
		Science and technology	Proportion of R&D cost in GDP (%)	D18	2.25	3.04	2.2	1.54	2.72	2.59	2.71	1.84	2.45
			Total number of R&D (thousand persons)	D19	75.38	69.69	30.86	8.58	16.80	39.06	39.08	25.09	26.79
		Social stability and security	Registered urban unemployment rate (%)	D20	2.04	3.00	2.81	2.33	2.15	3.23	2.93	3.00	2.07
			Number of doctors per thousand persons	D21	3.70	2.97	2.88	2.11	2.80	2.51	2.53	2.37	2.24
			Retirement pay per capita (thousand RMB)	D22	329.1	290.1	295.5	326.7	378	310	313.5	323.7	299
	Resource condition	Resources conservation	Water resources per capita (m3)	D23	165.87	31.55	151.66	118.36	238.79	250.92	118.46	168.09	141.41
			Area of cultivated land per capita (m2)	D24	504.4	574.8	451.9	610.9	1056.5	636	857.9	732.4	650.2
			Area of afforestation per capita (m2)	D25	16.84	11.07	21.34	30.89	17.04	19.34	22.71	18.36	26.21
		Resource consumption	Electricity consumption per capita (MWh)	D26	3.70	3.76	7.00	3.16	11.63	6.21	4.68	3.27	3.09
			Water consumption per capita (m3)	D27	204.85	89.59	222.53	145.44	452.95	123.89	138.83	280.27	194.44
		Resource utilization efficiency	Energy yield (thousand RMB/tce)	D28	144.4	178.2	80.2	98.5	160.1	182.5	124.1	142.7	135.8
			Water resource yield (RMB/m3)	D29	417.54	1141.11	399.83	360.11	360.95	741.78	401.44	172.53	290.03
	Ecological environment	Environmental state	Ammonia-nitrogen emissions per unit GDP (t/billion RMB)	D30	14.8	13.3	13.2	25.7	11.1	18.1	30.1	32.2	28.3
			COD emissions per unit GDP (t/billion RMB)	D31	176.6	157.5	140.8	242.5	184.1	217.6	312.8	326.1	344.7
			SO2 emissions per unit GDP (t/billion RMB)	D32	163.4	99	454.2	348.9	144.1	129.1	236.1	309.1	253.1
			NOx emissions per unit GDP (t/billion RMB)	D33	150.2	111.3	296.8	344.3	114.2	135.9	210	368.5	187.5
			Soot emissions per unit GDP (t/billion RMB)	D34	178.1	44.6	261.5	177.8	20	60	128.6	171.4	73.2
			Industrial solid waste emissions per unit GDP (t/billion RMB)	D35	0	0.16	1.74	-0.42	0	3.47	1.31	2.3	-3.76
		Natural ecological protection	Area of public park per capita (m2)	D36	10.5	14.2	18.4	15.1	25	21.1	18	14.3	20.2
Object Layer	System Layer	Characteristic Layer	Indicator Layer	No.	Value of Indicators
					Weihai	Rizhao	Laiwu	Linyi	Dezhou	Liaocheng	Binzhou	Heze
Regional sustainable development	Economic development	Economic scale	GDP per capita (thousand RMB)	D1	106.9	58.1	49.4	36.7	48.1	44.7	61.2	28.4
			Proportion of public finance budget in GDP (%)	D2	8.32	7.28	7.54	7.54	6.64	6.6	8.67	7.4
			Total fixed assets investment per capita (thousand RMB)	D3	90.7	48.9	45.8	31.2	39	35.2	51.6	12.6
		Economic structure	Proportion of the tertiary industry in GDP (%)	D4	45.4	42.9	40.4	46	40.3	37.1	41.9	36
		Economic development rate	Growth rate of GDP (%)	D5	8.5	7.5	6.57	7.1	7.6	8.79	7.08	9.26
			Growth rate of the output value of the tertiary industry (%)	D6	9.5	7.4	7.12	8.4	9.1	7.9	7.95	10.13
		Export-oriented economy	Total export-import volume per capita (million US$)	D7	6.04	5.27	1.40	0.85	0.55	0.85	2.11	0.51
		Economic benefit	Industrial added value per capita (thousand RMB)	D8	45.73	24.75	22.71	13.65	21.29	21.3	27.23	13.11
			Proportion of high and new technology industry output value in total designated size enterprises (%)	D9	38.83	20.12	19.85	27.21	28.05	26.26	26.26	31.61
	Social progress	Population development	Urbanization rate (%)	D10	63.16	54.81	58.84	53.82	51.73	46.15	54.62	45.13
			Number of students in ordinary universities for every thousand persons	D11	25.19	21.99	7.32	6.75	8.93	12.06	13.42	5.75
			Number of books in public libraries per capita	D12	0.63	0.26	0.37	0.27	0.25	0.19	0.35	0.15
		Living standard	Disposable income of urban residents per capita (thousand RMB)	D13	36.34	26.22	30.22	28.63	21.04	21.57	28.39	20.37
			Disposable income of rural residents per capita (thousand RMB)	D14	16.31	12.32	13.71	10.83	11.27	10.51	12.73	9.80
		Infrastructure construction	Number of urban public transport vehicles for every million persons	D15	59	32	81	17	10	22	28	10
			Area of urban road per capita (m2)	D16	33.41	30.41	29.08	23.01	32.95	27.28	21.99	22.27
			Density of the drainage pipes in built-up areas (km/km2)	D17	19.17	14.63	9.04	14.12	7.92	13.91	12.73	8.74
		Science and technology	Proportion of R&D cost in GDP (%)	D18	2.35	1.28	2.57	2.13	1.46	2.2	2.73	1.32
			Total number of R&D (thousand persons)	D19	20.71	6.44	7.09	22.33	12.99	14.16	20.91	9.52
		Social stability and security	Registered urban unemployment rate (%)	D20	1.54	2.00	2.49	2.35	2.80	3.04	2.19	3.18
			Number of doctors per thousand persons	D21	2.56	1.91	2.32	1.69	2.02	1.79	2.29	2.17
			Retirement pay per capita (thousand RMB)	D22	252.1	215.4	253.8	283.6	316.2	321	281.1	352.5
	Resources condition	Resources conservation	Water resources per capita (m3)	D23	221.72	234.38	133.92	209.18	216.64	111.04	275.46	242.11
			Area of cultivated land per capita (m2)	D24	694.6	833	536.9	816.1	1121.1	946.1	1207.1	977.6
			Area of afforestation per capita (m2)	D25	25.57	24.22	32.55	21.62	24.09	20.66	34.33	16.68
		Resource consumption	Electricity consumption per capita (MWh)	D26	3.74	5.90	7.37	3.44	3.47	6.54	2.49	2.11
			Water consumption per captia (m3)	D27	143.3	171.53	200.5	159.92	331.92	299.3	388.44	256.11
		Resource utilization efficiency	Energy yield (thousand RMB/tce)	D28	160.3	66.1	43.5	134.4	126.2	108.9	59.0	113.0
			Water resource yield (RMB/m3)	D29	746.66	338.22	245.69	228.21	144.33	149.06	157.13	110.29
	Ecological environment	Environmental state	Ammonia-nitrogen emissions per unit GDP (t/billion RMB)	D30	14	26.6	30.4	41.9	43.5	33.8	32.7	50.5
			COD emissions per unit GDP (t/billion RMB)	D31	101	257.1	270	387.1	535.5	533.1	562.2	550
			SO2 emissions per unit GDP (t/billion RMB)	D32	139.4	352.2	1103.3	270.6	292.8	268	414.3	379.1
			NOx emissions per unit GDP (t/billion RMB)	D33	133.2	342.5	886.2	291.4	210.8	307.4	475.3	316
			Soot emissions per unit GDP (t/billion RMB)	D34	56	700	2310.9	338.1	174.8	81.9	159.9	259.7
			Industrial solid waste emissions per unit GDP (t/billion RMB)	D35	0	0.1	-1.59	0.02	0	-0.01	17.72	-0.01
		Natural ecological protection	Area of public park per capita (m2)	D36	26.1	23.2	22.7	19.1	24.9	12.6	18.9	12.6

in the Appendix A.

3.3. Subsystem Assessment with Back Propagation Artificial Neural Network

The network training parameters for the subsystems are listed in

Table 3. The network training parameters of subsystems.

Name	Number of Nodes			Objective Precision	Times of Training	Learning Rate	Transfer Function		Training Function
	Input Layer	Hidden Layer	Output Layer				Hidden layer	Output layer
Economic subsystem	9	9	1	1 × 10−4	1000	0.01	logsig	purelin	traingdx
Social subsystem	13	12	1	1 × 10−4	1000	0.01	tansig	purelin	traingdx
Resources subsystem	7	7	1	1 × 10−4	1000		logsig	purelin	trainlm
Environmental subsystem	7	7	1	1 × 10−4	1000		logsig	purelin	trainlm

. According to Equations (5)–(9), the training results are described in Figure 5. The simulink model is used to simulate the trained network. The network output for different subsystems is listed in

Table 4. Network output results of different subsystems.

Cities	Economic Subsystem	Social Subsystem	Resources Subsystem	Environmental Subsystem
Weihai	3.51	3.61	4.32	4.55
Qingdao	4.43	4.74	3.15	4.24
Yantai	3.93	3.21	4.17	4.04
Dongying	3.67	4.03	2.65	4.42
Jinan	4.38	3.26	2.43	3.52
Linyi	3.23	2.53	3.60	2.76
Rizhao	2.52	2.61	3.33	3.39
Weifang	2.74	2.88	2.92	3.43
Tai’an	2.07	2.89	2.87	3.48
Zaozhuang	2.12	2.99	2.83	3.15
Dezhou	1.67	2.41	3.47	3.08
Zibo	2.78	2.71	1.72	3.76
Jining	2.38	2.46	2.43	2.83
Binzhou	0.86	2.97	2.38	2.22
Liaocheng	1.08	2.27	2.16	2.65
Heze	0.68	1.78	2.67	2.24
Laiwu	1.25	2.82	1.83	1.92

, which represents the scores of different subsystems in each city. The Economic, Social, Resources or Environmental level of each city can be compared separately. A higher score represents a higher level of development in this subsystem.

3.4. Comprehensive Evaluation for Regional Sustainable Development by Analytic Hierarchy Process with Entropy Correction

The weight of different subsystems was calculated by AHP with entropy correction through Equations (10)–(13), see

Table 5. The weights of different subsystems.

Subsystems	Weight
Economic subsystem	0.1028
Social subsystem	0.2681
Resources subsystem	0.3146
Environmental subsystem	0.3146

. The overall score of regional sustainable development level for the 17 different cities can be obtained through their overall score and weight for the four subsystems, see

Table 6. The overall score of regional sustainable development levels of different cities.

Weihai	Qingdao	Yantai	Dongying	Jinan	Linyi	Rizhao	Weifang	Tai’an
4.09	4.05	3.89	3.66	3.31	3.06	3.04	3.03	2.90
Zaozhuang	Dezhou	Zibo	Jining	Binzhou	Liaocheng	Heze	Laiwu
2.82	2.78	2.74	2.49	2.35	2.11	1.97	1.94

and Figure 6.

4. Discussion

4.1. Construction of Indicator System

To construct the indicator system for regional sustainable development, it was necessary to follow principles of hierarchy, simplicity, comprehensiveness, and operability. But it is difficult to ensure both the comprehensiveness and simplicity of the indicator system at the same time. Among the indicators with variation coefficients less than 0.12, only d6 was retained, due to its characteristic to reflect economic development. In each subsystem, the retained indicators were more representative than those deleted in discrimination analysis.

One of two related indicators will be deleted by correlation analysis if the correlation coefficient ≥0.8, but the remaining one may be deleted in the discrimination analysis, which leads to incomplete information being reflected by the indicators. Therefore, discrimination analysis should be given priority. Compared with the indicator systems such as the United Nations Sustainability Development Goals and China Sustainable Development Strategy Report, this indicator system applied multiple methods, such as correlation analysis, discrimination analysis, back propagation neural network, and AHP with entropy correction, and can reflect regional sustainable development more comprehensively and concisely.

4.2. Back Propagation Artificial Neural Network and Analytic Hierarchy Process with Entropy Correction

For nonlinear mappings, the sigmoid transfer function is usually applied for hidden layer and output layer. The output values of the entire network may be arbitrary. Three error curves in Figure 5 represent the training sample, validation sample, and test sample, respectively. The neural network automatically divides training samples into three types according to the default values of 75%, 15%, and 15%. The error of all three samples was very low, which means the neural network simulation was good. It also shows that ANN can realize the fitting of arbitrary nonlinear functions, which is consistent with the characteristics of the complex nonlinear system of regional sustainable development.

In order to ensure the consistency of the judgment matrix, the weights of different subsystems were calculated by AHP with entropy correction. The Economic, Social, Resources, and Environmental subsystems were weighted as 0.1028, 0.2681, 0.3146, and 0.3146, respectively. The economic development of Shandong Province has been rapid, but this also has also led to serious resource consumption and environmental pollution. The larger weight for the Resources subsystem and the Environmental subsystem shows these two subsystems are more important than the other two subsystems.

4.3. Analysis of Sustainable Development Levels

The overall score of regional sustainable development levels shows that Weihai and Qingdao have a very high level of regional sustainable development. Yantai, Dongying, Jinan, Linyi, Rizhao, and Weifang have a high level. Tai’an, Zaozhuang, Dezhou, Zibo, Jining, Binzhou, and Liaocheng have a general level. Heze and Laiwu have a low level.

Table 4 shows that coastal cities have comparatively high scores for different subsystems. The development of subsystems is coordinated, and economic and social development will not cause serious negative impacts on the Resources and Environment subsystems. As an inland city, the geographical advantage of Jinan is not obvious, but its Economic and Social development levels are better because it is the capital city of Shandong Province. The Resources subsystem is the main limiting factor affecting sustainable development for Qingdao, Jinan, and Dongying, since it is lower than the other three subsystems in these cities. For other inland cities, the four subsystem levels of Binzhou and Liaocheng are generally low, and the levels of Laiwu and Heze are even lower than these latter two cities. The constraints of Laiwu are shortage of resources and low environmental quality. For Heze, the levels of economic and social development are low, and the level of environmental quality also needs to be improved.

The sustainable development levels of different cities in Shandong Province present the following characteristics. Firstly, regional sustainable development shows a gradually decreasing trend from east to west and from coast to inland (see Figure 6). Secondly, coastal cities have higher regional sustainable development capacity, due to their geographical advantages. Their foreign trade capabilities are strong, and their economic development does not rely on heavy industries. Their proportions of tertiary industry are large and environmental quality is superior. Thirdly, cities such as Jining, Liaocheng, Dezhou, and Heze, which suffer from poor economic development, usually lag behind in social development. This is reflected in aspects such as residential living standards, educational attainment levels, urban infrastructure construction, levels of medical care, science and technology, and social security. In order to improve the level of social development, these cities need to show improvement in education, technology, and urban construction. At the same time, accelerating economic development and providing financial support for social development are further issues. Fourthly, environmental pollution is serious in economically backward cities. In these cities, the industrial structure is dominated by secondary industry. More efforts should be made to enhance science and technology development, optimize industrial structure, and reduce pollutant emissions.

This research provides a new integrated indicator system and evaluation model for regional sustainable development, which can provide a useful reference for developing policies to increase the capability of regional sustainable development in Shandong Province. Compared with the 234 indicators in the United Nations Sustainability Development Goals, this indicator system with 36 indicators and the evaluation model with correlation analysis, discrimination analysis, back propagation neural network, and AHP with entropy correction can reflect regional sustainable development more comprehensively and concisely. This methodology is applicable for different cities or different provinces in a country, but it is not suitable for making comparison between countries, because some indicators may represent different factors in different countries. Additionally, the data for some indicators are not available in some countries. In the future, a common index system that is suitable for evaluating national sustainable development needs to be explored.

5. Conclusions

An indicator system for regional sustainable development was established using the processes of initial indicator selection, discrimination analysis, and partial correlation analysis, which includes 3 layers, 4 subsystems, and 36 indicators. BP ANN was used to calculate the scores of the four subsystems. The overall score of the regional sustainable development of the 17 cities was evaluated by AHP with entropy correction. The results show that the cities with a very high level of sustainable development are Weihai and Qingdao, and those with a low level are Heze and Laiwu. The sustainable development levels of these 17 cities in Shandong Province show a gradually decreasing trend from east to west and from coast to inland. Cities with an underdeveloped economy usually have backward social development and serious environmental pollution. Through the indicator system and the scores of subsystems, the restrictive factors of the sustainable development level of each city were identified. These findings provide a useful reference for developing policies to increase the capability of regional sustainable development.