A Decomposed Data Analysis Approach to Assessing City Sustainable Development Performance: A Network DEA Model with a Slack-Based Measure

Abstract

This paper deals with urban sustainable development in China. We propose a network data envelopment analysis (DEA) model with a slack-based measure (SBM) to analyze the eco-efficiency of 284 Chinese cities, enabling us to find a way to open the “black box” in conventional DEA models and introduce social well-being factors into the model, and depict the role of local government in providing public service and improving social well-beings. We set up a framework of urban development by dividing the process of into two steps. The first stage is a production system translating inputs and natural resources into GDP and waste production, which will be inputs to the second stage for distribution and consumption to realize social welfare and environmental protection. The results show eco-efficiency of Chinese cities experienced a significant decrease from 2005 to 2016, which should be mainly attributed to the distribution and consumption processes. Structural differences are described by regions, administrative level and clusters. These results are compared with an existing urban sustainability index system developed by McKinsey and an ANOVA approach is conducted to reveal differences between cities across regions and clusters. This article sheds new light on the understanding of urban sustainable construction and development in China regarding the service performance of local government.

1. Introduction

Urban sustainable development has increasingly attracted people’s attention, particularly after the United Nations announced in 2015 its sustainable development goals “to make cities and human settlements inclusive, safe, resilient and sustainable” [1]. It stated that more than two-thirds of the world’s population would be living in cities by 2050. With the world’s largest population, China has experienced dramatic growth in its urbanization rate in the past 40 years, when more than 400 million people moved into cities and became part of the urban population. However, it has been reported that many Chinese cities have inadequate water supplies and uncontrolled waste management problems [2,3], and scholars have become concerned about the extent to which China’s economic growth has benefited its citizens’ well-being, with local governments leading the process of urban development [4]. Individuals have started to realize that the development of a city is about more than economic growth. Conventional GDP per capita can no longer be the sole measurement of urban development. However, benchmarking the performance of urban development across dimensions that relate to the sustainability of the economy, energy, water, environment, and other types of systems requires a holistic approach. Accordingly, the research community has proposed different methods including analytic hierarchy processes (AHP) [5], indicator systems [6,7] and fuzzy evaluation methods [8] to measure the development of cities.

Eco-efficiency, initially described by Schaltegger and Sturm [9], is the ability to produce goods and services while minimizing inputs including natural resources and negative environmental impacts [10], and has been thought to be a good attempt to reflect the performance of urban development. Data envelopment analysis (DEA) has attracted great attention based on a total-factor production process to measure eco-efficiency [11]. However, in the context of urban development, there are weaknesses in the conventional DEA model that cannot be ignored. First, urban sustainability is a systematic process with a variety of both desirable and undesirable indicators in multiple stages of development. Overlooking the internal structure, conventional DEA models usually treat such systems as “black boxes” and do not explicitly consider undesirable indicators. Second, previous studies have generally evaluated eco-efficiency as a single process, focusing on the industrial sector, which directly generates waste, while omitting the fundamental role of government [12]. The government sector is significant, as it provides public goods and services for a sustainable production environment. How the government sector affects the industrial sector and how it allocates outputs in the production sector, as well as the efficiency in different stages, is still unknown. These facts and studies motivated us to use advanced DEA models to assess urban sustainable development, especially by revealing the internal relationship between the “black box” of urban construction and social welfare benefits.

In this paper, we apply a network DEA model with a slack-based measure (SBM) to 284 Chinese cities to assess their eco-efficiency. We define eco-efficiency as the efficiency of a city’s economic and social well-being development given the resource and environmental capacity constraints it faces. A network structure of the two-stage system is constructed to reveal a comprehensive structural measure of urban development. Each region is regarded as a two-stage network structure in which the first stage is characterized as the production system and the second stage as the consumption and distribution system. Indicators on different dimensions are suggested across this system after identifying inputs and outputs among different stages.

The contribution of this paper to the existing literature is as follows: first, most conventional DEA models consider the urban development system to be a “black box” and ignore the internal relationships among the different stages of urban development. This article sheds new light on the exploration of models attempting to integrate a clear structure into the urban development process to reflect internal interactions among different stages of production and social distribution. The role of the government in urban sustainable development is discussed in a new way, concentrating on the duty of public service provision to the citizenry—which is the elementary and final unit of the city—rather than to manufacturers and enterprises. Second, most current approaches are based on economic growth with constraints on environmental sustainability and energy use, with GDP considered their sole final output. In such analyses, the importance of social well-being indicators has, thus far, been underestimated. Therefore, we include public service indicators to reflect the performance of urban social welfare, such as numbers of doctors and students as well as pension coverage and green coverage, to reveal the social well-being of urban citizens, which we regard as the final output instead of GDP.

2. Literature Review

Urban sustainability is perhaps the most popular subject in sustainability development studies and has been analyzed in a variety of settings. Considering the broad definition of urban sustainability, researchers have used different perspectives on this topic, including analytic hierarchy processes (AHP) [5], indicator systems [6,7] and fuzzy evaluation methods [8]. However, these approaches have been criticized for lacking an objective standard of variable selection or being inaccurate and unconvincing because of data standardization or panel data co-integration.

Among all these researchers, eco-efficiency (initially described by Schaltegger and Sturm [9]) is considered an approach intended to promote a transformation towards sustainability [13]. Data envelopment analysis (DEA) has become one of the main techniques for measuring relative efficiency due to the seminal work of Charnes et al. [14]. With the increasing use of DEA, scholars noticed some weaknesses in the conventional DEA models while utilizing them in regional sustainable development. One of the challenging issues encountered in sustainability assessment is how to include in any model undesirable outputs, which are inevitable during the process of urban development and operation, such as carbon dioxide (CO₂), sulfur dioxide (SO₂), and other pollutants. Based on the seminal study by Tone and his coauthors [15,16], researchers have sought to widen the use of DEA methods for evaluating eco-efficiency by proposing an SBM for DEA to measure system efficiency [17,18,19]. Yin et al. [20] measured the eco-efficiency of 30 Chinese cities with a super-efficiency DEA model in which environmental pollution is considered an undesirable output, showing that almost half of the provincial capital cities are eco-efficient. Sueyoshi and Yuan [21,22] applied DEA models to assess municipal sustainable development in China incorporating undesirable outputs. Huang et al. [23] created composite eco-efficiency indicators by integrating eco-efficiency with economic, energy, and environmental efficiencies and used an SBM-DEA approach to assess the eco-efficiency of 30 Chinese provinces. Yu et al. [24] developed three DEA models that incorporated meta-frontier and undesirable outputs, including SBM, EBM, and MinDW and compared them among these three measures; an empirical approach measuring the eco-efficiency of 117 Chinese prefecture-level cities was also included. Ding et al. [25] proposed a non-radial and multi-objective generalized DEA model for economic–environmental efficiency evaluation to empirically measure the economic–environmental performance of Chinese provincial regions to illustrate their improved accuracy. As for the selection of variables in specific DEA models while evaluating urban sustainable development, capital, labor, energy consumption, and land used are commonly considered to be inputs for production, while GDP and the value added in industry are widely used desirable outputs, and CO₂ emissions, SO₂ emissions, and waste water discharge are commonly included as undesirable outputs [26,27].

Another weakness of the DEA models that has been widely criticized is that they utilize multiple inputs to produce multiple outputs, while urban sustainability development is a systematic process with a variety of indicators in multiple stages of development, with complex interactions and systematic structure. Overlooking these internal structures, conventional DEA models treat such systems as a one-stage step, which means that all inputs are used and all outputs are generated in a single process, regarding the production process as a “black box”, and therefore they are unable to reflect the relationships between the variables inside a given process [28]. In order to cope with such a weakness, several modified versions of DEA were put forward, such as the relational or centralized model and the non-cooperative model. These facts and studies motivate us to use advanced DEA models to assess urban sustainable development, especially by revealing the internal relationship in the “black box” of urban construction and social welfare benefits. Following the early attempt of taking the overall system into the component processes introduced by Charnes et al. [29], who found that army recruitment had two processes—that is, creating awareness through advertisements and creating contracts—measuring the efficiency by separating a large system into detailed smaller production processes has attracted increasing attention and has been adopted in many research fields [30,31].

Network DEA models were introduced to measure the eco-efficiency of countries, regions, and cities. Wu et al. [32] proposed a two-stage network DEA model to assess China’s regional energy saving and emission reduction efficiency, where the second stage disposes of the pollutants produced in the first stage to make it a two-stage structure. Li et al. [33] applied a DEA model in a two-stage setting to measure the regional economic and ecological efficiency of 31 Chinese administrative regions where the first stage modeled the ecological system and the second stage modeled the decontamination system, which included recycling and treating undesirable outputs coming from the first stage. Zhang et al. [34] adopted a three-stage DEA model to eliminate interference from the external environment and statistical noise and utilized it to assess the industrial eco-efficiency levels of 30 Chinese provinces. They concluded that their average industrial eco-efficiency level increases by 30% after eliminating the impacts from the external environment and statistical noise. Xiao et al. [27] present a two-stage network DEA framework incorporating government and industrial sectors and measured the eco-efficiency of 84 resource-based cities. However, the existing literature mainly focuses on the process of energy utilization and production, and there have been few articles paying enough attention on the process of urban social benefit and welfare distribution.

The literature above made great progress on the measurement of eco-efficiency, especially in China, using different approaches, including DEA. However, there are still some aspects that need improvement and consideration. First, the selection of the appropriate dimensions of urban development for modeling the system processes is critical for successful urban planning with the appropriate emphasis and connotation. The existing literature mainly uses GDP as the desirable output or even the final output in network DEA models, while this variable still needs distribution and consumption to truly benefit the citizens in the region or industry of concern. Second, although some scholars are beginning to consider the internal structural relationship between regional sustainable development and eco-efficiency, there still seem to be few studies that provide a systematic approach to reveal the process of urban sustainable development in China, especially taking the role of the local government in terms of public service provision into consideration. Given the challenge of providing a comprehensive model that takes the above perspectives into account, this study describes a convincing structure to develop a two-stage network DEA model for urban sustainable development process evaluation.

3. Framework and Formulation

3.1. Subsection Framework of Urban Sustainable Development

In this paper, we develop a framework to investigate the process of urban sustainable development, which is divided into two steps, i.e., economic production and distributive consumption, as is shown in Figure 1. The first step is to use the inputs and resources each city possesses, such as capital, population, and natural resources, to produce economic outputs, with constraints imposed on the natural resource use and environmental capacity. Then, in the second step, the economic outputs produced in the first stage, including government revenues and industrial waste emission and control, will be introduced as inputs to benefit social well-being and protect the urban environment. The eco-efficiency throughout the entire process is then evaluated as the weighted average of the sub-efficiencies of both steps.

The reason and foundation of this two-step framework of urban sustainable development can be described and summarized as follows:

First, China has experienced dramatic economic growth since market reforms and openness that began in 1978. Among all these tasks, economic growth was fundamental and most important, influencing the performance measurements and promotions of local officials during these years. It is common knowledge that there is a competition among local Chinese officials and the most important evaluation criteria is the rate of local economic growth [35]. Although the Chinese central government has attempted to improve the performance appraisal of local officials [36], the real results of this still lack empirical evidence.

Additionally, there is a consensus that Chinese local governments have played an irreplaceable role in the process of urban sustainable development such as by investment promotion, infrastructure development, social welfare coverage, and environmental protection. Local governments make their own efforts and contribution by deeply engaging in the process of economic development including through public service supplements, industrial development funds, and land supply. Besides, local public services including health, education, and infrastructure are mostly dependent on the investment and maintenance of the local government. However, the fiscal decentralization reform that began in 1994 changed the fiscal structure and conditions of Chinese local governments, which have faced federal budgetary constraints while being responsible for providing most of the basic public goods and services to support urban sustainable development [37]. Such fiscal decentralization also promoted competition among local governments. The implementation of urban sustainable development is highly relevant to the financial input of local governments, which is closely related to economic development and the growth of cities, and has been encouraging Chinese urban economic development [38]. On the one hand, taxes that were divided into local income components included business and income taxes, which are closely related to local economic development performance. On the other hand, government officials also had to seek additional sources of revenue such as land transfer income and local government debt, which are also highly related to the condition and development of the local economy in a balanced budget. Therefore, the provision of public goods by the local government, which significantly influences welfare and urban construction, is highly dependent on the development of the local economy. Besides, there has also been criticism that local governments in China have preferred providing public service sectors which could benefit intermediate economic development while overlooking sectors needing long-term investment, such as education, health, and the environment, especially in rural regions [4]. Some scholars also criticized the low efficiency of fiscal expenditure of local governments in China [39]. Therefore, we set up our urban development model as a two-step system that includes economic development and municipal infrastructure processes. Economic development is a sine qua non prerequisite of the second step, which utilizes the positive achievements and copes with the negative output produced in step 1.

3.2. Network DEA Model with a Slack-Based Measure

We propose a slack-based measure (SBM) network DEA model to assess the efficiency of city resource allocation in pursuing sustainable development. The inputs and outputs, both desirable and undesirable, are assigned into different stages of the system, making the weights reflect the efficiency and returns on resource investments. Then, we evaluate the performance of these factors against the city’s sustainable development level using classification techniques. For the model described below, we make use of the following symbols and definitions:

$E_o^k$	Efficiency associated with division k
$x_{i0}^{k,g}$	Desirable input variables of division k
$y_{r0}^{k,g}$	Desirable output variables of division k
$y_{r0}^{k,b}$	Undesirable outputs variables of division k
$s^{k,g-}$	Slack variable on the desirable input of division k
$s^{k,g+}$	Slack variable on the desirable output of division k
$s^{k,b+}$	Slack variable on the undesirable output of division k
${\mathsf{\Lambda }}^k$	Intensive vector after linearization in division k, where ${\mathsf{\Lambda }}^k=t^k{\lambda }^k$
${\lambda }^k$	Intensive vector corresponding to the inputs and outputs in division k
tk	Variable for linear transformation of division k
$p^k$	Predefined weights to represent the contribution of division k to the efficiency of the system

We consider a system composed of d divisions and m inputs and s outputs in all d divisions. For each division, we choose a slack-based measure with desirable and undesirable variables, as follows [15]:

$$\underset{}{E_o^k{}_{}=\min }\frac{1-\frac{1}{m_1}{{\displaystyle \sum }}_{i=1}^{m_1}\frac{s_i^{k,g-}}{x_{io}^{k,g}}}{1+\frac{1}{s_1+s_2}\left({{\displaystyle \sum }}_{r=1}^{s_1}\frac{s_r^{k,g+}}{y_{ro}^{k,g}}+{{\displaystyle \sum }}_{r=1}^{s_2}\frac{s_r^{k,b+}}{y_{ro}^{k,b}}\right)}$$

(1)

s.t.

$$x_0^{k,g}=X^{k,g}{\lambda }^k+s^{k,g-}$$

$$y_0^{k,g}=Y^{k,g}{\lambda }^k-s^{k,g+}$$

$$y_0^{k,b}=Y^{k,b}{\lambda }^k+s^{k,b+}$$

$$s^{k,g-}\ge 0, s^{k,g+}\ge 0,s^{k,b+}\ge 0, {\lambda }^k\ge 0$$

Model (1) is a nonlinear programming model that can be transformed to a linear programming model as shown below:

$$E_o^k{}_{}=\underset{}{\min }t-\frac{1}{m_1}{{\displaystyle \sum }}_{i=1}^{m_1}\frac{s_i^{k,g-}}{x_{io}^{k,g}}$$

(2)

s.t.

$$1=t^k+\frac{1}{s_1+s_2}\left({{\displaystyle \sum }}_{r=1}^{s_1}\frac{s_r^{k,g+}}{y_{ro}^{k,g}}+{{\displaystyle \sum }}_{r=1}^{s_2}\frac{s_r^{k,b+}}{y_{ro}^{k,b}}\right)$$

$$x_0^{k,g}{t}^k=X^{k,g}{\mathsf{\Lambda }}^k+s^{k,g-}$$

$$y_0^{k,g}{t}^k=Y^{k,g}{\mathsf{\Lambda }}^k-s^{k,g+}$$

$$y_0^{k,b}{t}^k=Y^{k,b}{\mathsf{\Lambda }}^k+s^{k,b+}$$

$$s^{k,g-}\ge 0, s^{k,g+}\ge 0,s^{k,b+}\ge 0,{\mathsf{\Lambda }}^k\ge 0,t^k>0$$

Models (1) and (2) are both for non-oriented divisional efficiency. For the input-oriented divisional efficiency, our model becomes the following:

$$\underset{}{E_o^k{}_{}=\min }[1-\frac{1}{m_1}{{\displaystyle \sum }}_{i=1}^{m_1}\frac{s_i^{k,g-}}{x_{io}^{k,g}}]$$

(3)

s.t.

$$x_0^{k,g}=X^{k,g}{\lambda }^k+s^{k,g-}$$

$$y_0^{k,g}=Y^{k,g}{\lambda }^k-s^{k,g+}$$

$$y_0^{k,b}=Y^{k,b}{\lambda }^k+s^{k,b+}$$

$$s^{k,g-}\ge 0, s^{k,g+}\ge 0,s^{k,b+}\ge 0,{\lambda }^k\ge 0$$

Similarly, for output-oriented divisional efficiency, the model becomes the following:

$$\underset{}{E_o^k{}_{}=\min }\frac{1}{1+\frac{1}{s_1+s_2}\left({{\displaystyle \sum }}_{r=1}^{s_1}\frac{s_r^{k,g+}}{y_{ro}^{k,g}}+{{\displaystyle \sum }}_{r=1}^{s_2}\frac{s_r^{k,b+}}{y_{ro}^{k,b}}\right)}$$

(4)

s.t.

$$x_0^{k,g}=X^{k,g}{\lambda }^k+s^{k,g-}$$

$$y_0^{k,g}=Y^{k,g}{\lambda }^k-s^{k,g+}$$

$$y_0^{k,b}=Y^{k,b}{\lambda }^k+s^{k,b+}$$

$$s^{k,g-}\ge 0, s^{k,g+}\ge 0,s^{k,b+}\ge 0,{\lambda }^k\ge 0$$

After we obtain the efficiencies of the individual stages, we apply the efficiency aggregation model (5) presented in [40] to calculate the systematic efficiency.

$${\tilde{E}}_0={{\displaystyle \sum }}_{k=1}^d{p}^k{E}_0^k$$

(5)

4. Research Design

4.1. Variable Selection

The system process of urban sustainable development shown above can be divided into two steps. We chose capital investment, population, electricity, land, and water as traditional inputs [41,42,43]. Electricity, land, and water variables are proxies for the natural resources used in urban development and should be considered for long-term sustainable development. As for the outputs of the first stage, they should reflect not only the achievement of economic production processes but also the industrial waste produced in the first process. Therefore, traditional GDP, as a desirable economic output, and industrial waste, dust production, and industrial SO₂ production, as the undesirable outputs, are introduced into step 1. Recognizing the crucial role of the local government in the urban development process, which is mainly in charge of local pensions, health, and education systems, we introduced in this stage the available government revenues.

Industrial wastewater emissions, industrial dust emissions, and SO₂ emissions are entered into the model as undesirable outputs in step 1. These outputs, both desirable and undesirable, are the results of the economic production process and should be regarded as intermediate products, which are outputs in step 1 and inputs in step 2 and will be utilized and disposed of in the second stage of this network structure.

Regarding the outputs in step 2, since social well-being is a dimension that has many aspects, the research community has not yet reached an agreement on a precise definition. According to the National Bureau of Statistics of China, national general public expenditure is mainly on education (14.61%), urban and rural community affairs (10.43%), agriculture, forestry, and water conservation (9.57%), healthcare (6.98%), transport (4.94%), and science and technology (3.99%) [44]. Based on the distribution of public finances, we adopted the urban sustainability index [7]. This study chose the number of middle school students, people receiving a pension/unemployment insurance, and buses as outputs of the second step. For undesirable outputs, the emission of industrial dust and industrial sulfur dioxide are introduced as outputs in step 2 to reflect the result of waste treatment and disposal in urban development.

While applying the DEA analysis, there should be no indication of multicollinearity among the variables introduced into the model. However, some abovementioned variables may reflect similar economic implications, thus they might present a relatively high correlation. Principal component analysis is used to reduce the dimensionality of the variables to cope with this issue. For example, the variables pertaining to construction land area, water consumption, and electricity consumption all reflect the use of natural resources to pursue economic growth and urban development. Thus, they show high correlations with each other. Similar conditions also appear among the outputs in the second stage, such as the number of pensions and unemployment benefits dispersed, doctors, buses, and green areas, representing the development of the social welfare system and residential facilities. Therefore, three compound variables are introduced into the model to help reflect the corresponding economic connotations. More details about the data selection and corresponding processing are listed and described in

Table 1. Summary of input and output indicators.

Category	Preference	Variable	Unit/Calculation Method
Input		Total investment in fixed assets	CNY 100 million
Input		Population	10 thousand persons
Input		Natural resources	PCA by construction land, water and electricity consumption
Intermediate	Desirable	Gross domestic product	CNY 100 million
Intermediate	Undesirable	Industrial dust production	Tons
Intermediate	Undesirable	Industrial SO2 production	Tons
Output	Desirable	Number of middle school students	10 thousand persons
Output	Desirable	Insurance covered	Average of number of pension and unemployment insurance covered
Output	Desirable	Social welfare	PCA by public transit, number of doctors and green area
Output	Undesirable	Industrial dust emission	Tons
Output	Undesirable	Industrial SO2 production	Tons

Source: China City Statistical Yearbook (National Bureau of Statistics of China 2006–2017).

.

The inclusion of many variables in different dimensions naturally prompts questions of variable selection, decomposition of influence, and analysis of main effects for inputs and outputs. Global sensitivity and uncertainty analysis (GSUA) consider the probability distribution functions of all variables and their nonlinearity dependency on interactions. This should be done for any tracked indicator pattern or for a composite systemic indicator that combines them and has been updated to enhance the practicality of this method [45,46,47]. GSUA can extract drivers of patterns also considering their interdependencies. Since there is an uncertainty that needs to be characterized, it seems necessary to utilize a sensitivity analysis to verify the stability and robustness of model construction [48,49,50].

4.2. Data Sources and System Settings

As for the other aspects in this model, the time range from 2005 to 2016 was due to data availability. There are 284 Chinese cities in our database, containing more than 85% of China’s population. The data sources included the China City Statistical Yearbook and the China City Construction Statistical Yearbook. There were some missing values for some variables, such as the emission of SO₂ due to regulation. When less than 10% of the values were missing, we used a random forest method to estimate them. Considering the network DEA model is a process that combines economic development and natural resource utilization, we chose an input orientation during the whole process.

The network DEA model was coded and solved by CPLEX 12.4 on the Dell Latitude E7470 with an Intel i7 2.20 GHz processor. The statistical analysis of the results was coded in R. To compare the performance of the measurement, we classified the cities based on their characteristics and eco-efficiency levels and then compared the classification results with the results obtained from the USI system, where the weight is determined by the assessments of experts.

5. Results and Discussion

5.1. Results

The eco-efficiencies for the 284 Chinese cities are shown in the map below, based on their location. Their colors reveal the eco-efficiency and sub-efficiency according to our calculations. Our sample covered most Chinese cities, especially those in the eastern and interior region. Figure 2 presents the result of sub-efficiency of step 1 in 4 different observation years. The first row shows the situations in 2005 and 2009, while the second row presents those in 2013 and 2016. In these pictures green is the most evident color, indicating that the performance of cities remains to some extent stable during the whole period. However, there is still some structural evolution at this stage. In the western area, the number of yellow and red cities, even in some eastern and coastal regions, tends to decrease, especially in the second decade of the 21st century. Only metropolitan cities such as Beijing, Shanghai, and some northeastern cities realize efficiency improvement in economic development during these 12 years.

Figure 3 shows the results of sub-efficiency in step 2 for different years. In 2005, most Chinese cities had a sub-efficiency value of more than 0.4, causing the main colors of that figure to become yellow, orange, or even red, while the decreasing trend is quite obvious. In 2016 there were no more than ten cities whose color was not green, indicating that cities all around China have experienced a significant decline in their sustainable consumption and distribution the urban development process. This confirms that most Chinese cities did not perform better on municipal construction to make them more livable and sustainable. Only regions such as Beijing, Yangtze River Delta, Pearl River Delta, and some northeastern cities retained a relatively higher level of sub-efficiency at the end of the observation period.

Summarizing both steps, the process efficiency of urban sustainable development could be calculated using the given algorithm. The results of eco-efficiency analysis are presented in Figure 4. It can be seen that in the second stage, most Chinese cities experienced a decline in efficiency, the most significant of which is evident in the western and interior regions. Eastern and coastal cities also have decreased their efficiency to some extent, while most developed and promising municipalities such as Beijing, Shanghai, Guangzhou, and Shenzhen still retained high standards of eco-efficiency over time, especially in the 2010s.

Thereafter, the calculations can be described in aggregate terms. While computing the average of a certain group, we used population to obtain the weighted average to reflect the average efficiency of the whole sample or certain group. The results are presented in

Table 2. Eco-efficiency and sub-efficiency of cities in China during 2005 and 2016.

Year	Step 1	Step 2	Eco-Efficiency
2005	0.359875	0.708473	0.504845
2006	0.350035	0.628946	0.468639
2007	0.346354	0.558494	0.437005
2008	0.37506	0.463279	0.409947
2009	0.329887	0.445033	0.378022
2010	0.326863	0.398745	0.355874
2011	0.386908	0.386287	0.379682
2012	0.377242	0.339053	0.352661
2013	0.373844	0.338867	0.350461
2014	0.37976	0.339067	0.35331
2015	0.389094	0.282634	0.328346
2016	0.402664	0.228865	0.30524
Average	0.366466	0.426479	0.385336

. The eco-efficiency of Chinese cities declined during our observation period and the reason can be mainly attributed to the decline of sub-efficiency in step 2 of urban construction and pollution treatment. In step 1, which mainly reveals the efficiency of economic development, a slow increase was seen over the 12-year period, even though there was some fluctuation around 2008 as a result of the global financial crisis. As for the second stage, the sub-efficiency in this step experienced a dramatic decline and the downward trend is very stable, from more than 0.7 in the first year to only 0.28 at the end of the observation period in 2016. The reason for this decline might be attributed to the selection of variables in our model. The outputs in step 2 included variables of insurance coverage and number of middle school students, which were naturally increasing at a slower rate than GDP. It might also indicate that increases in social welfare development in Chinese cities was far behind the overall rate of economic growth, and the pollution treatment efficiency in most cities still lagged in comparison with modern metropolitan areas such as Shenzhen.

To highlight the findings of this study, the results are reported by representative samples or in groups instead of as a detailed city-to-city comparison. The results for individual cities are available in the Appendix A. These Chinese cities can be grouped in terms of dimensions such as administrative levels (municipalities, provincial capital, vice-provincial cities, as well as prefecture-level and county-level cities. Our sample did not include county-level cities because of data availability), regions (eastern, northeastern, interior, and western regions) and different city clusters or groups.

The four municipalities included Beijing, Tianjin, Shanghai, and Chongqing, and these four cities have the highest administrative levels in China. Among the fifteen vice-provincial cities, ten of them (Shenyang, Changchun, Ha’erbin, Nanjing, Hangzhou, Jinan, Wuhan, Chengdu, Xi’an, and Guangzhou) are capital cities of their corresponding provinces, and the other five (Shenzhen, Dalian, Qingdao, Ningbo, and Xiamen) are specially designated in the state plan as key cities. These 19 cities might be able to benefit from resources coming from the central government to help promote urban sustainable development. Figure 5, Figure 6 and Figure 7 report the average sub-efficiency and eco-efficiency levels of the different administrative city groups during the 2005–2016 period.

In Figure 5, the economic growth sub-efficiency levels of the cities in different administrative levels remain almost the same in the first half of the period, while showing significant differences compared to those of prefecture cities in the second half of the period, which had an even higher economic growth sub-efficiency level at the beginning. However, in the second half of the period, cities with higher administrative levels performed better than cities with lower levels, confirming the economic growth efficiency in Chinese cities of various administrative levels. Prefecture-level cities did not make any improvement on efficiency during the observed period, while the efficiency values of the municipalities and vice-provincial cities in 2016 were almost double the efficiency values in 2005.

As for the sub-efficiency values of the second stage, Figure 6 shows that the cities at different administrative levels all experienced dramatic declines during the whole period, while cities with higher administrative levels still performed better than the prefecture cities at the end of the observation period. Municipalities and vice-provincial cities even realized some rebound during the years of 2011 to 2012. Prefecture cities were in the first place among the three administrative levels initially but ended up in third place in 2016, the end of our observation period. These two sub-efficiency coefficients led to the final and combined eco-efficiency result, which is presented in Figure 7, and it can be described as the combination of the results of both steps. Thus, cities with higher administrative levels showed significant advantages, which were mainly revealed in the second decade of the 21st century. This administrative advantage did make a great difference regarding the urban sustainability levels of the Chinese cities.

As for regional groups, most studies have conventionally divided China into the eastern, interior, and western regions. However, these divisions can no longer meet our needs due to changes in the real conditions. Recently, the National Bureau of Statistics started to utilize a four-group classification criterion. With this, the provinces and municipalities in China can be grouped by location into the following four regions: eastern, interior, western, and northeastern, as shown in

Table 3. Regional classification of provinces in China.

Region	Provinces or Municipalities Included
Eastern	Beijing, Tianjin, Hebei, Shanghai, Jiangsu, Zhejiang, Fujian, Shandong, Hainan
Interior	Shanxi, Anhui, Jiangxi, Henan, Hubei, Hunan
Western	Inner Mongolia, Guangxi, Chongqing, Sichuan, Guizhou, Yunnan, Tibet, Shaanxi, Gansu, Qinghai, Ningxia, Xinjiang
Northeastern	Liaoning, Jilin, Heilongjiang

.

Figure 8, Figure 9 and Figure 10 show the population weighted average efficiency values of the cities in different Chinese regions during our observation period. There are significant differences among regions in terms of the sub- and eco-efficiency perspectives, and the regions took different paths during the 16 years. Interior and western cities showed some fluctuation during the 12 years but experienced little change without any advance overall. On the other hand, the northeastern and eastern cities had a relatively higher starting point in step 1 and widened their advantage over the interior and western cities during the observation period. For step 2, the sub-efficiency values of all four regions were declining, while these two regions started at a relatively low standard among the four regions but also ended up with the top two places in 2016, indicating that eastern and northeastern cities were also performing more efficiently while utilizing their economic outputs to carry out urban construction. According to the results, eastern and northeastern cities showed higher combined eco-efficiency levels at the aggregate level at the end year, and their advantage seemed to expand in the later years, which might be mostly due to their performances in the economic efficiency section. Starting from the last rank, the eastern area became one of the regions with the highest efficiency by the end of 2016. It is well known that unbalanced regional development has become an issue that is receiving increasing attention in China. The government made great efforts and attempts to promote development in the central and western parts of the country by introducing preferential policies and devoting significant resources. However, our calculations indicate that the efficiency levels of these regions continue to lag behind, and these gaps seem still to be widening in recent years and could do so even in the future.

There is another trend that should be stressed regarding Chinese urban development, which is that the central government has been making a great effort to promote the formation and development of city agglomeration. The central government has approved development plans for more than 10 national-level city groups since 2018, and these covered more than 50% of the cities in our sample and more than 80% of the national population. Therefore, it is important to analyze the eco-efficiency results obtained from the perspective of city groups and clusters. Since there have been more than 10 city groups under planning, we chose the three largest and most important city groups in our sample (Beijing–Tianjin–Hebei, Yangtze River Delta, and the Guangdong–Hong Kong–Macao Greater Bay Area) to compare their eco-efficiency and sub-efficiency results.

Figure 11, Figure 12 and Figure 13 present the population weighted average results of the eco-efficiency and sub-efficiency calculations of the cities in the three largest clusters. It can be observed that the Guangdong–Hong Kong–Macao Greater Bay Area had the highest level of development and efficiency on urban sustainability among these three city groups and among all of the 13 Chinese city groups in terms of both economic growth and urban sustainable construction. Regarding the economic growth process, although there was some fluctuation around 2011, all three city groups still improved the efficiency of their economic development at the end of the observation period compared to the beginning of 2005. The second step and the combined eco-efficiency figures presented a similar condition, with the Big Bay Area being the most efficient group and the other two at almost the same level. We should also point out that these city groups were the top three according to our results and comparisons.

Since the selection of variables in our approach mainly referred to USI, it is natural to compare the results of our eco-efficiency and USI scores. We should point out that the two evaluation systems were built based on different frameworks. The USI approach calculates a weighted average of the output variables after standardized processing, which emphasizes the results of urban development without considering how much cost or input it takes to obtain the current level of achievement. The eco-efficiency measure in our study pays more attention to the efficiency of transferring inputs to outputs. This approach is important for evaluating urban sustainable development and observing the performance of Chinese cities with respect to both evaluation systems. The results are presented in

Table 4. Comparison of eco-efficiency and USI scores (2005–2016 average).

City	Eco-Efficiency	USI	City	Eco-Efficiency	USI
Beijing	0.7932	0.7258	Shanghai	0.6293	0.7204
Tianjin	0.4538	0.6277	Chongqing	0.3257	0.4263
Chengdu	0.3959	0.5734	Dalian	0.3857	0.6125
Fuzhou	0.3363	0.5346	Guiyang	0.3282	0.6134
Guangzhou	0.5508	0.7460	Ha’erbin	0.3795	0.5043
Haikou	0.9710	0.6073	Huhehaote	0.3747	0.5796
Hangzhou	0.4902	0.7016	Hefei	0.3269	0.5408
Kunming	0.3288	0.6075	Lanzhou	0.3800	0.5774
Jinan	0.3920	0.6050	Nanchang	0.3656	0.5643
Nanjing	0.4172	0.6894	Qingdao	0.3864	0.5756
Nanning	0.3021	0.4886	Ningbo	0.4119	0.6153
Xiamen	0.5470	0.7953	Xining	0.3658	0.5030
Shenzhen	0.9550	0.8137	Shenyang	0.4224	0.5973
Wuhan	0.3814	0.6465	Changchun	0.3828	0.5827
Changsha	0.4353	0.5647	Zhengzhou	0.3270	0.5827
Shijiazhuang	0.3056	0.5162	Urumqi	0.5060	0.6524
Xi’an	0.3692	0.6050	Yinchuan	0.3382	0.5957

. For simplicity, we only report the result of the municipalities, vice-provincial cities, and provincial capital cities.

In Table 4, metropolitan cities such as Beijing, Shanghai, Guangzhou, and Shenzhen, which are widely accepted as first-tier Chinese cities, performed well in terms of both the final score and from the process efficiency perspective, confirming their leading position throughout China. This is particularly true of Shenzhen, which ranked in first place in both lists, making it the city with the highest sustainability efficiency level. Big cities in coastal provinces (Nanjing, Hangzhou, Tianjin) and some tourist cities (Xiamen and Haikou) also performed well on the USI, with a relatively high efficiency in making full use of the resources to maximize their “revenue” from urban sustainable development (eco-efficiency). However, the eco-efficiency and USI results are not always in agreement. There were also some cities (Xi’an, Yinchuan, Guiyang), mainly in western China, that obtained a relatively good USI score with low eco-efficiency, indicating that their urban development is, to some extent, inefficient. Some interior and western cities (Chongqing, Shijiazhuang, and Nanning) showed an even poorer performance on both the USI score and eco-efficiency.

5.2. Analysis of Variance

The analysis above shows the difference in eco-efficiency and sub-efficiency among Chinese cities in different groups and between the final USI scores and process efficiency in our approach. These simple comparisons, however, might be insufficient. A case in point is that the group analysis cannot reflect the individual results of all 284 cities in our sample, implying that further analysis is necessary to make our analysis more convincing and comprehensive. Therefore, we used ANOVA to compare the results of both approaches.

We performed an ANOVA to verify the difference in sub-efficiency and eco-efficiency among regions and other city groups. The results show that there is a significant difference (p < 0.05,

Table 5. Results of ANOVA on sub-efficiency and eco-efficiency.

Section	Group	Df	Sum of sq	Mean of sq	F Value	Pr(>F)
Step 1	Region	3	4.593	1.53091	43.387	2.2 × 10−16
	City groups	9	8.718	0.96862	28.377	2.2 × 10−16
Step 2	Region	3	3.975	1.3250	25.332	3.3 × 10−16
	City groups	9	6.594	0.73266	14.192	2.2 × 10−16
System	Region	3	2.656	0.88527	37.886	2.2 × 10−16
	City groups	9	5.591	0.62121	27.555	2.2 × 10−16

) among different regions independent of the sub-efficiencies and eco-efficiency section, verifying that our analysis above is convincing and robust. We grouped our cities into 10 city groups and performed another ANOVA. The results obtained show that the difference among groups is statistically significant and the cities in different groups show different characteristics in sub-efficiencies and eco-efficiency.

Our next task was to validate our analysis of the comparison between eco-efficiency and the available USI scores. For this purpose, we developed a randomized complete block design for validation, dividing our sample into different blocks to see whether there was any difference between USI scores and eco-efficiency figures in each block. We compared the means using a Duncan multiple range test at the 95% confidence level. The results in

Table 6. Results of randomized complete block design among regions.

Group	Df	Sum of Squares	Mean of Squares	F Value	Pr(>F)
Eastern	629	23.904	0.0380	2.443	2 × 10−16
Interior	429	4.091	0.0095	1.547	3.39 × 10−16
Western	409	7.698	0.01882	1.417	0.000223
Northeastern	139	1.2703	0.00914	0.927	0.671243

indicate that among all four regions, only the scores and eco-efficiency levels of eastern cities had significant differences, while for the other regions no statistically significant differences could be observed. That means there might be a mismatch between the USI score and eco-efficiency in eastern cities, i.e., they may get a good USI score with a relatively low eco-efficiency or vice versa. However, such patterns did not repeat in the other three regions. The reason for these results might be that the east region is the most developed and has higher investment levels than other regions. Governments of eastern cities might not be able to make full use of these investments and city government in other regions did not have to manage with such investment pressure. Accordingly, the result of the randomized block design might not be significant. There is no significant difference between the USI score and eco-efficiency when we used administrative level as a block. Such a mismatch did not exist in this cluster, indicating that larger cities were able to handle their input and investment into urban development efficiently just as smaller cities.

5.3. Sensitivity Analysis

Based on our explanation above, we conducted a sensitivity analysis to help us evaluate the importance and influence of variables included in our model. The results are presented in Appendix A. We should point out that the selection and position of each variable was based on the framework of urban sustainable development in the theoretical analysis. Therefore, the sensitivity analysis provides a peripheral reflection on the weight and importance of the variables in the model, but should not be a criterion of model construction.

More importantly, we only present average values over time while skipping confidence intervals, which limits the information that can be provided. Therefore, this information can be plotted considering the probability distribution function of variables, and the result has been presented in Figure 14. With most plots distributed around 0 value, the robustness and stability of our model has been confirmed and there would be little probability that fluctuation of a single variable would drastically change our calculations and the efficiency of the urban sustainability construction and development. We also present a robustness check in Appendix A where the sample has been divided into three parts, and the results remain stable, confirming the suitability of the model and the sensitivity of each variable to influence the calculations.

6. Conclusions and Policy Implications

6.1. Conclusions

To cope with several weaknesses in the conventional DEA model, an SBM network DEA model was introduced to assess the performance of urban sustainable development in 284 Chinese cities, measuring the eco-efficiency of transferring factor inputs into economic growth, social well-being, and preventing waste emissions. An ad hoc framework reflecting urban development was divided into a production section and distribution section to reveal phased features of each step. For the selection of variables, a series of social well-being indicators was included to reveal the benefit of citizens from developing processes.

We presented our results by comparing the eco-efficiency levels of cities in different groups. We also reported the comparison analysis between results in this approach and USI to reveal the difference in the sustainability final score and eco-efficiency. An analysis of variance and a sensitivity analysis were conducted to reflect the heterogeneity of eco-efficiency and importance of each variable included in the model, providing other perspectives to understand the results of eco-efficiency calculated in our model. Through the research, we discovered some important characteristics of eco-efficiency.

(1) The eco-efficiency of Chinese cities experienced a significant decrease during our observation period from 2005 to 2016 while the efficiency of economic development, to some extent, remained stable. The main reason for this eco-efficiency decrease should be attributed to the distribution and consumption processes. This means that economic development of most Chinese cities was not well transferred into improvements in social welfare coverage and environment protection, resulting in a decreasing trend on eco-efficiency in general.

(2) When we focus on structural characteristics of eco-efficiency, we can see dramatic differences among cities of different administrative levels, geographic regions and city groups. Cities in higher administrative levels and east regions (eastern and northeastern) tended to strengthen their advantages during this period. Among the three most important city groups, the performance of Big Bay Area is significantly better than that of the other two.

(3) We compared the result of eco-efficiency, which pays more attention to the efficiency, and USI index, which emphasizes the results of urban development. First-tier cities performed well in terms of both aspects, while we can also see structural differences between the two index results in some interior and western cities, indicating an inefficient urban sustainable development.

6.2. Policy Implications

Based on the results obtained, some policy implications are summarized as follows.

Firstly, Chinese cities should be vigilant about the decreasing efficiency of transforming economic development into tangible improvement in welfare coverage, urban infrastructure development, and reduction of pollution emissions. This is of particular importance considering the enormous pressure coming from the mass migration of more than 200 million individuals from rural to urban areas. The China’s urbanization process will be further extended and deepened. Local governments should pay more attention to the efficiency of urban infrastructure construction to ensure that economic achievement could benefit every citizen including rural populations.

Secondly, it is crucial for the central government to recognize the imbalance among cities in different administrative levels, geographic regions, and city groups. Larger cities have better demonstrated their stronger ability and urban construction efficiency than small ones. In order to realize a long-term sustainable development, the central government should continuously commit to maintain the balance among cities and reduce their efficiency gaps by further sharing prosperity and promoting communication to enable the spillover of knowledge and management ability among cities, especially within the same city group. In addition, it is also important to point out that there is a significant difference between urban development and efficiency, especially in cities in the interior and western regions, suggesting stakeholders must improve input–output efficiency while pursuing urban development.

There are several directions for future research. It is imperative to open the “black box” of urban sustainable development to reveal this complex process, which can be divided in different ways to reflect its diversification and intricacies. Second, some other data variables related to eco-efficiency and urban sustainability are not reported in the yearbook published by the Chinese government. They can also be introduced into this model to help reveal the efficiency of this process. Regardless of some data limitations, it is worthwhile to attempt presenting a two-step process-based framework using a network DEA model with an SBM to analyze the compositional data on urban development and to classify cities based on their uses of resources for sustainable development. The outcomes of this study can help city managers, investors, and other stakeholders evaluate investments and address shortcomings in future development planning.