Efficiency Evaluation of Regional Environmental Management Systems in Russia Using Data Envelopment Analysis

Abstract

The concept of eco-efficiency has recently become an issue of great importance due to the growing trend of environmental degradation, and many approaches based on Data Envelopment Analysis (DEA) are used in the literature to evaluate the environmental performance of economic systems. However, research to date has paid little attention to the possibility of extending the DEA approach to the problem of measuring the economic feasibility of eco-efficiency improvement. The main aim of this study is to evaluate the efficiency of investments focused on improving the eco-efficiency of the regional economy in Russia using the DEA approach. The various types of costs for environmental protection measures are considered as inputs and the annual decrease in specific environmental impacts of the regional economy are considered as outputs of DEA models. This is different from previous research, which generally focused on environmental efficiency only, omitting the integration of economic aspects in eco-efficiency measures. This study compares three different modifications of basic DEA models in the context of technical complexity and practical feasibility. The results show that the efficiency of regional environmental management in many Russian regions has a great potential for improvement. From a practical point of view, the Slack-Based Measure (SBM) model provides the most accurate results for policy applications. Unlike other ratings, the DEA-SBM model may stimulate an optimization of environmental protection spending and the introduction of technological and organizational eco-innovations.

1. Introduction

The concept of eco-efficiency, first introduced in 1990 by Schaltegger and Sturm [1], has become an issue of great importance due to the growing need for reducing the global trend of environmental degradation. Because of its multidimensional nature, measuring the eco-efficiency of production systems with the help of Data Envelopment Analysis (DEA) in recent years has become the mainstream in the scientific literature [2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]. In most studies, environmental efficiency (or eco-efficiency) is considered as the ability of a production system to minimize undesirable outputs (negative externalities of production activity) with fixed inputs (material and human resources) and maximize desired outputs (indicators of the economic performance of the production system) [2,5,6,9,10,11,13]. Therefore, DEA models summarize various desirable and undesirable effects of the production system in a single efficiency index. As a rule, eco-efficiency is evaluated regionally [2,5,6,7,10,14] or sectorally [8,9,11,12,13]. Such inputs of Decision-Making Units (DMUs) as energy (volume of energy consumption), labor (number of employed), capital (volume of investment in fixed assets), land (amount of fertile land used), or water (volume of freshwater consumption) are often considered as inputs. Gross regional product or the gross output of the industry are considered as desirable outputs, while the volumes of various types of pollution as undesirable outputs. Efficiency scores of DMUs, calculated by DEA, provide valuable information for formulating policies for more sustainable development of economical systems under the study.

However, research to date has paid little attention to the possibility of extending the DEA approach to the problem of measuring the economic feasibility of eco-efficiency improvement. Only a few studies include the investments in minimization of negative environmental impacts into considerations while designing DEA model [7,16,17]. These previous papers have provided important information on some technical problems that can arise in the process of calculation of efficiency scores in such extended models. For instance, if we consider improvements in eco-efficiency (which appear due to investments) as outputs of the DEA model, in reality for some DMUs they can be negative. If we apply a two-stage model for production systems eco-efficiency estimation on stage 1 and efficiency of investments in pollution treatment measurement on stage 2, we face the problem of aggregation of two efficiency scores in one objective function.

In addition to the above-mentioned DEA-characteristic issues, we must also note that the basic models for eco-efficiency calculations found in the literature also have several flaws. First, basic DEA models do not put any restrictions on the weights of inputs and outputs. Therefore, in many cases equally effective DMUs can have an opposite burden on the environment: one can produce the minimum amount of emissions into the atmosphere, then the other produces, for instance, the minimum amount of solid waste. However, these DMUs can be severely sub-optimal in other environment categories. Second, basic DEA models can evaluate too many DMUs as efficient ones. In other words, they have poor discrimination of efficiency. Due to these features, such models are insufficient to investigate realistic problems and not helpful for the formulation of policy by local governments or industry leaders.

Therefore, despite the fact that extensive research has been carried out on measuring eco-efficiency with DEA-framework, there is still a lot of room for improvement.

The purpose of this study is twofold. The first aim is to evaluate the efficiency of investments focused on improving the eco-efficiency of the regional economy using DEA approach. In this study, as inputs we consider various types of costs for environmental protection measures: (1) investments in fixed assets aimed at reducing pollution and (2) current costs for environmental protection measures. As outputs, we consider the annual decrease in specific environmental impacts of the regional economy. This is different from previous research, which generally focused on environmental efficiency only, omitting the integration of economic aspects in eco-efficiency measures. From a practical perspective, the purpose of the study is to evaluate the efficiency of environmental management at the regional level. The second aim of the study is to compare three different modifications of basic DEA models in the context of technical complexity and practical feasibility. As modifications, we propose to eliminate the problem of unequal consideration of various environmental aspects in the overall assessment of the efficiency of the regional environmental management by introducing weights. Moreover, we also consider two possible ways to work with negative outputs.

The rest of this paper is structured as follows. The literature review is presented in Section 2. Section 3 introduces the models and methods used for efficiency evaluation. Section 4 describes the data and variables, and then reports the results of DEA efficiency evaluation. Finally, Section 5 provides discussions on the results and their main implications.

2. Literature Review

In environmental management, the decision process plays a vital role in implementing policies to promote development based on eco-efficient economic growth [18,19,20,21,22,23,24]. In this regard, for benchmarking and environmental performance assessment different approaches are used, which include parametric (stochastic frontier analysis) [25,26,27,28], nonparametric (data envelopment analysis), and multicriteria decision-making (MCDM) [29,30,31,32,33] approaches. In some studies on environmental efficiency assessment, MCDM methods are combined with DEA, e.g., Zhou et al. [34] used the TOPSIS method for assessment of inputs and outputs indicators and then studied the spatiotemporal variation of the comprehensive eco-efficiency of 21 cities. Martín-Gamboa et al. [35] presented a review of life cycle approaches coupled with data envelopment analysis within multicriteria decision analysis for sustainability assessment of energy systems. Keshavarz and Toloo [36] proposed a hybrid DEA and multi-attribute decision-making (MADM) approach to sustainability assessment.

Nevertheless, for environmental efficiency evaluation, as confirmed by the literature mentioned in this review, the DEA approach has become more common in recent years. There exist a number of excellent literature reviews devoted to the analysis of environmental efficiency studies based on DEA. Song et al. [4] investigate the achievements of the theoretical and practical basis of environmental policy analysis using DEA. Sueyoshi et al. [37] systematically summarize previous research efforts on DEA applied to energy and environment from the 1980s to 2010s, including concepts and methodologies on DEA environmental assessment. Mardani et al. [38] reviewed and classified 145 papers devoted to the application of DEA models in the assessment of environmental and energy economics. Xu et al. [39] provide a literature review on energy efficiency evaluation based on DEA. This paper deals indirectly with environmental efficiency evaluation, including energy efficiency evaluation of regions, and therefore is also of some interest.

However, since these reviews were published, a number of researches have appeared in the scientific literature. Therefore, this review is mainly focused on recent papers. Moreover, in this section, we have reviewed articles devoted principally to the study of regions. The papers mentioned in our literature review are summarized in

Table 1. Summary of the recent literature on environmental efficiency evaluation of regions using DEA.

Ref.	DMUs	RTS	DEA Model and Methodology	Application Area
[40]	31 provinces in China	CRS	SBM model and factor analysis	Water use
[41]	27 EU countries	CRS	Two-stage efficiency measurement based on DDF model	Environmental performance
[42]	16 Polish regions	CRS	CCR model and Hellwig’s method and coefficient of determination	Environmental efficiency
[43]	30 regions in China	CRS	SBM model based on undesirable outputs	Environmental efficiency of industry
[44]	24 Mediterranean countries	VRS	SBM model and two-stage double bootstrap	Energy efficiency
[45]	30 regions in China	VRS	SBM model and Tobit regression and truncated regression models	Energy-environmental performance
[46]	30 provinces in China	VRS	Three-stage analysis based on BCC model & SFA	Industrial eco-efficiency
[47]	31 provincial administrative regions in China	CRS	Two-stage US-SBM model & spatial autocorrelation analysis using Moran’s I	Water resource utilization efficiency
[48]	30 Chinese provinces	CRS	Meta-frontier Super-US-SBM model	Composite eco-efficiency indicators
[49]	16 Polish regions	VRS	BCC model	Life cycle assessment
[50]	30 provinces in China	CRS	Intermediate approach and performance indices	Social sustainability
[34]	21 cities in Guangdong province, China	CRS	Super-SBM and panel regression models	Estimation of eco-efficiency
[51]	108 countries	CRS	Slack-based models with undesirable outputs	National environmental efficiency
[52]	28 EU countries	VRS	SBM model and dynamic DEA	Energy efficiency
[53]	30 provinces in China	VRS	Models with strong and weak disposability of undesirable outputs and Malmquist–Luenberger index	Environmental efficiency
[54]	17 regions of Central Federal District, Russia	CRS	CCR model and Malmquist index	Ecological-economic efficiency
[55]	30 regional industry systems in China	CRS	Two-stage CCR model and regression analysis	Environmental efficiency
[13]	30 provinces in China	CRS	SBM model and GML productivity index and panel Tobit model	Air pollution emission efficiency
[56]	30 Chinese provinces	VRS	Three-stage DEA approach based on BCC model and SFA	Energy efficiency
[57]	48 cities in Bohai Rim, China	CRS	Super-US-SBM model and Moran’s I	Eco-efficiency
[58]	282 European regions	VRS	Hyperbolic distance function measure and metafrontier approach	Eco-efficiency
[59]	28 EU countries	CRS	Environmental Efficiency Index	Environmental efficiency
[60]	30 provinces of China	VRS	Super-PEBM model and window analysis	Green economic efficiency
[61]	14 prefecture-level cities in Guangxi Province, China	CRS	Meta-US-SBM model and Tobit Model	Mining industry eco-efficiency
[62]	27 EU Countries	CRS	CCR model and fractional regression model	Eco-efficiency
[63]	30 regions in mainland China	VRS	Dynamic network DEA approach based on the SBM model	Eco-eficiency
[64]	30 provinces in China	CRS	Super-US-SBM model and regression analysis	Eco-efficiency of industrial investment
[65]	30 province-level regions of Chinese mainland	VRS	Two-stage DEA approach based on BCC model	Energy and environment efficiency

Notes. RTS: returns to scale; CRS: constant returns to scale; VRS: variable returns to scale; SBM: slack-based measure; DDF: directional distance function; US-SBM: SBM with undesirable outputs; Super-SBM: SBM with super efficiency; Super-US-SBM: SBM with undesirable outputs and super efficiency; Super-PEBM: epsilon based measure based on Pearson correlation coefficient; Meta-US-SBM: meta-frontier SBM with undesirable outputs and super efficiency.

.

Russian regions have large variability in their area and population. The differences become more considerable after the territorial reform 2003–2008 which led to the merging of regions in East Russia. It must be recognized that the area of some regions in Russia exceeds many European countries. Therefore, the literature review is limited not only by studies of the regions, but also includes papers where the environmental efficiency of countries is evaluated. Jebali et al. [44] examine the energy efficiency determinants in the Mediterranean countries. A number of papers [41,52,59,62] evaluate the evolution of environmental performance in member countries of the European Union (EU). Grigoroudis and Petridis [51] present a study of environmental efficiency measurement that concerns 108 countries from all over the world.

Several studies are devoted to the analysis of the environmental efficiency of cities. Such papers have also attracted some interest as large cities are often considered autonomous territorial units under the administrative division. Zhou et al. [34] estimated the eco-efficiency of 21 cities in Guangdong province, China. Li et al. [61] carried out mining eco-efficiency measurement for 14 prefecture-level cities in Guangxi Province, China. Zhou et al. [57] discuss eco-efficiency of 48 cities in Bohai Rim mainly focused on the role of industrial structure upgrades. Perhaps the most extensive study (by the number of regions) considered here was carried out by Bianchi et al. [58] and covers 282 European regions at NUTS-2 level.

As for the territorial aspect, note that most of the recent studies investigate the regions of China; the EU countries are in second place. The environmental efficiency of Russian regions has not been sufficiently studied using the DEA approach; only a few works there exist in the DEA literature in this area. In addition, these studies are limited, as they did not consider all the regions, but only those included in the Central Federal District [54] and Southern Federal District [66].

Usually, the inputs and outputs of the DEA model are chosen a priori by researchers. They are mainly dictated by the purpose of the study and are determined by the availability of data sources. In [42], the authors use Hellwig’s method and coefficient of determination to ensure an appropriate combination of environmental and economic indicators of the eco-efficiency evaluation.

A second-stage analysis is often performed with the help of various regression methods. Shi [55] and Zhu et al. [64] use regression analysis to identify the relationship between the efficiency scores and its determinant factors. Deng et al. [40] explore the factors that influence water use efficiency with a panel data model. Zhou et al. [34], among other methods, also used a panel regression model to determine factors influencing eco-efficiency. In [13,45,61], Tobit regression and truncated regression models are applied. Moutinho and Madaleno [62] analyze the relationship of the obtained DEA scores from the possible influencing factors, namely eight different types of pollutants, with the help of a fractional regression model. Jebali et al. [44] in the second-stage analysis applied a parametric bootstrap procedure to the truncated regression of DEA bias-corrected efficiency scores on environmental variables. Furthermore, in the second stage, Zhao et al. [47] and Zhou et al. [57] calculated a Moran’s I that represents a measure of spatial autocorrelation.

Often, researchers break down the eco-efficiency evaluation in several stages. Zhang et al. [46] adopted a three-stage approach to eliminate the interference of the external environment and statistical noise. Gómez-Calvet et al. [41] conducted a two-stage analysis which develops environmental performance indicators in the first stage for each pair country–year and evaluates its evolution in the second. Zhao et al. [56] employed a three-stage DEA approach combining BCC model [67] and the stochastic frontier analysis (SFA) model.

In the network DEA model, the efficiency measurement process is also split into several stages. Zhao et al. [47] use two-stage network DEA to consider undesirable output: the first stage describes the water utilization process and the second stage characterizes the pollutant treatment process. Shi [55] considers the internal structure of the production system to evaluate the environmental efficiency in the same way, i.e., production subsystem and pollutant treatment subsystem. Zhu et al. [65] divided energy and environment efficiency analysis into two sub-stages: energy production sub-stage and energy utilization sub-stage. Wang et al. [63] proposed a dynamic network DEA approach that considers the dynamic features of network DEA. This approach takes into account that capital investments in the current period are used to establish fixed assets for two stages, which can also be used in the later period, thus affecting its two-stage eco-efficiency.

Most studies employ undesirable outputs as the assessment of environmental efficiency involves the use of indicators that have a negative impact on the environment and, therefore, should be reduced. The traditional radial (BCC and CCR models) and non-radial slack-based measure (SBM) models are commonly applied for environmental efficiency assessment (see, i.e., in [43,48,49]), but there exist some studies that use the directional distance function (DDF) methods [41]. Bianchi et al. [58] used a hyperbolic distance function measure that considers improvements on the input and output direction simultaneously. A newly proposed and rather interesting approach is proposed in [50]; Zhang et al. applied an intermediate model, which is a combination of radial and non-radial approaches.

In order to evaluate the efficiency change over time, several approaches were applied. In [54], dynamic problems of eco-efficiency are explored with the help of Malmquist index estimation. Piao et al. [53] accomplished dynamic environmental performance analysis using Malmquist–Luenberger (ML) index. Wang et al. [13] used the global Malmquist–Luenberger (GML) productivity index to overcome some difficulties with ML index. Lu et al. [52] applied the dynamic DEA model to evaluate the intertemporal eco-efficiency. Wang et al. [63] combined dynamic and network DEA approaches into one model to evaluate industrial eco-efficiency and assess internal ineffective sources. Window analysis is used in [50,60,66].

The vast majority of previous studies use absolute values of indicators in models to analyze environmental efficiency. The main difference between this work and others is that we do not use absolute values of indicators, but their increments as variables. This leads to the values of the indicators are no longer strictly positive and can now take negative values. As a result, calculations based on DEA models become more complicated, as negative values must be taken into account.

3. Methodology

3.1. DEA Background

DEA is a powerful tool to evaluate the relative efficiency of a set of homogeneous decision making units (DMUs). It was originated by Charnes, Cooper, and Rhodes [68] based on seminal work of Farrell [69], where the linear programming technique to estimate a production frontier was first developed. In the DEA model, the relative efficiency score of any DMU is determined as a measure of the relative improvements in inputs and outputs between the DMU and the efficient frontier. In order to construct the efficient frontier, a collection of DMUs is used. The efficiency score can be evaluated by minimization of input keeping its output levels constant (input-oriented model), or by maximization of output keeping the inputs at the same rate (output-oriented model).

Consider a set of n observed DMUs: $\left\{{DMU}_j|j=1,\dots ,n\right\}$. Suppose each DMU${}_j$ consumes a number of inputs $X_j={(x_{1j},x_{2j},\dots ,x_{mj})}^{\mathsf{T}}$ and produces multiple outputs $Y_j={(y_{1j},y_{2j},\dots ,y_{rj})}^{\mathsf{T}}$. It is assumed that vectors $X_j$ and $Y_j$ are non-negative and has at least one positive component. The production possibility set T is determined as a set $\left\{\right(X,Y\left)\right|$ the outputs $Y\ge 0$ can be produced from the inputs $X\ge 0\}$. The boundary of PPS represents the frontier which is used to evaluate DMUs against each other.

We start with the Charnes, Cooper, and Rhodes (CCR) model [68] which is one of the most popular DEA models [70]. From the literature review, we recognize that a CRS assumption is mostly used in studies at the region and country level. Besides its popularity, this model fits the features of our task, because output variables that we take for regional environmental management systems assessment express not absolute values but their growth rates. Therefore, the model with constant returns to scale is more suitable in this case. The production possibility set of the CCR model can be written in the form

$$T=\left\{(X,Y)\right|X\ge \sum _{j=1}^n{X}_j{\lambda }_j,Y\le \sum _{j=1}^n{Y}_j{\lambda }_j,{\lambda }_j\ge 0,j=1,\dots ,n\},$$

(1)

The output-oriented CCR model for efficiency evaluation of DMU${}_k$ is written as follows [71]:

$$\begin{array}{cc}\hfill max& \eta \hfill \\ \hfill s.t.& \sum _{j=1}^n{X}_j{\lambda }_j\le X_k,\hfill \\ \hfill & \sum _{j=1}^n{Y}_j{\lambda }_j\ge \eta Y_k,\hfill \\ \hfill & {\lambda }_j\ge 0,j=1,\dots ,n.\hfill \end{array}$$

(2)

Efficiency score for CCR model is evaluated as $1/{\eta }^{\ast }$, where ${\eta }^{\ast }$ is optimal objective value of model (2). From the model (2), it follows that ${\eta }^{\ast }\ge 1$ and, consequently, efficiency score of a DMU is from 0 to 1. The unit is considered efficient if the efficiency score is equal to one, and inefficient if less than one. The efficiency score may be seen as a distance from the DMU${}_k$ to the efficient frontier.

Another popular DEA model proposed by Banker, Charnes, and Cooper (BCC) [67] exhibits variable returns to scale and their output-oriented case can be written in the form

$$\begin{array}{cc}\hfill max& \eta \hfill \\ \hfill s.t.& \sum _{j=1}^n{X}_j{\lambda }_j\le X_k,\hfill \\ \hfill & \sum _{j=1}^n{Y}_j{\lambda }_j\ge \eta Y_k,\hfill \\ \hfill & \sum _{j=1}^n{\lambda }_j=1,\hfill \\ \hfill & {\lambda }_j\ge 0,j=1,\dots ,n.\hfill \end{array}$$

(3)

3.2. Weights Restrictions

Weights restriction are used in DEA in order to incorporate implicit judgments in the dual form for modeling production trade-offs. For model (2), the dual output-oriented CCR model can be written in the form

$$\begin{array}{cc}\hfill min& v^{\mathsf{T}}{X}_k\hfill \\ \hfill s.t.& v^{\mathsf{T}}{X}_j-u^{\mathsf{T}}{Y}_j\ge 0,j=1,\dots ,n,\hfill \\ \hfill & u^{\mathsf{T}}{Y}_k=1,\hfill \\ \hfill & v\ge 0,u\ge 0.\hfill \end{array}$$

(4)

The dual model (4) provides another way of looking at the problem (2). Dual variables u and v are often called weights in DEA, because the efficiency score of DMU is defined as the ratio of a virtual output (weighted sum of outputs ${\sum }_{i=1}^r{u}_i{y}_{ik}$) to a virtual input (weighted sum of inputs ${\sum }_{i=1}^m{v}_i{x}_{ik}$). In the model (4), dual variables are supposed to be non-negative, and therefore they can be equal to zero in the optimal solution. This means that some variables are ignored in the efficiency evaluation. In order to overcome this situation, it is proposed to insert weight restrictions into the model. A broad overview of DEA models with weight restrictions is presented in [72].

Many of the models proposed in the DEA literature assume that constraints are imposed on actual weights. However, the application of such models in practice is often difficult, as the cost prices and/or production costs are not always known. A more promising approach is when the constraints are defined as the ratio of the input/output weights to the virtual input/output [73].

$$L_s\le \frac{u_s{y}_{sj}}{{\sum }_{i=1}^r{u}_i{y}_{ij}}\le U_s.$$

(5)

A similar weights restriction could be applied to inputs. Note that constraints are DMU-specific. Therefore, such restrictions are added with respect to all the DMUs.

The resulting model, call it CCR-AR, is a generalization of the assurance region method [74,75]. In this case, the upper and lower bounds of the specified constraints show the relative importance of each indicator relative to the others. Another way to improve the frontier is described in [76]. This method is based on including artificial production units and rays in the primal space of inputs and outputs. An algorithm for improving the envelopment in the DEA models using this approach were proposed in [77].

3.3. Non-Radial Model

One of the generally recognized models for efficiency evaluation in the presence of negative indicators is the non-radial model. Currently, there exist a large number of different non-radial DEA models to cope with negative data, see, i.e., in [78,79]. In particular, Portela et al. [80] showed that the CCR model cannot be applied for datasets where negative data can exist and the other type of technologies with variable returns to scale assumption should be considered.

In this paper, we used the approach described in [81], hereinafter referred to as the SBM model. This model is quite general; it takes into account the presence of negative data, allows us to set weights for each variable, and also makes it possible to rank efficient DMUs using the super-efficiency method [82,83]. The model of Lin et al. [81] is adopted from the well-known RAM model [84] which is used the following input and output ranges:

$$\begin{array}{cc}\hfill R_i^{-}& =\underset{1\le j\le n}{max}\left\{x_{ij}\right\}-\underset{1\le j\le n}{min}\left\{x_{ij}\right\},i=1,\dots ,m,\hfill \\ \hfill R_s^{+}& =\underset{1\le j\le n}{max}\left\{y_{sj}\right\}-\underset{1\le j\le n}{min}\left\{y_{sj}\right\},s=1,\dots ,r.\hfill \end{array}$$

As a result, the following SBM model was proposed in [81]:

$$\begin{array}{cc}\hfill min& \frac{1+{\sum }_{i\in \mathbb{I}}{\mu }_i{w}_i^{-}/R_i^{-}}{1-{\sum }_{s\in \mathbb{O}}{\nu }_s{w}_s^{+}/R_s^{+}}\hfill \\ \hfill s.t.& x_{ik}\ge \sum _{j=1,j\ne k}^n{x}_{ij}{\lambda }_j-w_i^{-},i\in \mathbb{I},\hfill \\ \hfill & y_{sk}\le \sum _{j=1,j\ne k}^n{y}_{sj}{\lambda }_j+w_s^{+},s\in \mathbb{O},\hfill \\ \hfill & \sum _{j=1,j\ne k}^n{\lambda }_j=1,\hfill \\ \hfill & {\lambda }_j\ge 0,j=1,\dots ,n,j\ne k,\hfill \\ \hfill & w_s^{+}\le R_s^{+},r\in \mathbb{O},\hfill \\ \hfill & w_s^{+},w_i^{-}\ge 0,\forall s\in \mathbb{O},i\in \mathbb{I},\hfill \end{array}$$

(6)

where $\mathbb{I}=\left\{i|R_i^{-}>0,i=1,\dots ,m\right\}$, $\mathbb{O}=\left\{r|R_s^{+}>0,s=1,\dots ,r\right\}$, ${\mu }_i$, and ${\nu }_s$ are positive weights determined by decision-maker’s (DM) opinion and satisfying ${\sum }_{i\in \mathbb{I}}{\mu }_i=1$ and ${\sum }_{r\in \mathbb{O}}{v}_r=1$. Model (6) is more general than RAM model, as ${\mu }_i$ and ${\nu }_s$ can take any values while in RAM model they are equal to $1/m$ and $1/s$, respectively.

In model (6), the projection onto the frontier may not be Pareto efficient. Thus, a second stage should be accomplished in order to identify strongly Pareto efficient projection:

$$\begin{array}{cc}\hfill min& \frac{1-{\sum }_{i\in \mathbb{I}}{\mu }_i{s}_i^{-}/R_i^{-}}{1+{\sum }_{i\in \mathbb{O}}{\nu }_i{s}_i^{+}/R_i^{+}}\hfill \\ \hfill s.t.& x_{ik}=\sum _{j=1,j\ne k}^n{x}_{ij}{\lambda }_j-w_i^{-\ast }+s_i^{-},i\in \mathbb{I},\hfill \\ \hfill & y_{ik}=\sum _{j=1,j\ne k}^n{y}_{ij}{\lambda }_j+w_i^{+\ast }-s_i^{+},i\in \mathbb{O},\hfill \\ \hfill & \sum _{j=1,j\ne k}^n{\lambda }_j=1,\hfill \\ \hfill & {\lambda }_j\ge 0,j=1,\dots ,n,j\ne k,\hfill \\ \hfill & s_i^{-}\ge 0,i\in \mathbb{I},\hfill \\ \hfill & s_i^{+}\ge 0,i\in \mathrm{O}.\hfill \end{array}$$

(7)

Problems (6) and (7) are fractional and can be solved using Charnes and Cooper’s transformation [85] to the equivalent linear programming problems. Problem (6) is transformed to the following problem:

$$\begin{array}{cc}\hfill min& \left(t+\sum _{i\in \mathbb{I}}{\mu }_i{\widehat{w}}_i^{-}/R_i^{-}\right)\hfill \\ \hfill s.t.& t{x}_{ik}\ge \sum _{j=1,j\ne k}^n{x}_{ij}{\widehat{\lambda }}_j-{\widehat{w}}_i^{-},i\in \mathbb{I},\hfill \\ \hfill & t{y}_{sk}\le \sum _{j=1,j\ne k}^n{y}_{sj}{\widehat{\lambda }}_j+{\widehat{w}}_s^{+},s\in \mathbb{O},\hfill \\ \hfill & t-\sum _{s\in \mathbb{O}}{\nu }_s{\widehat{w}}_s^{+}/R_s^{+}=1,\hfill \\ \hfill & \sum _{j=1,j\ne k}^n{\widehat{\lambda }}_j=t,\hfill \\ \hfill & {\lambda }_j\ge 0,j=1,\dots ,n,j\ne k,\hfill \\ \hfill & {\widehat{w}}_s^{+}\le t{R}_s^{+},s\in \mathbb{O},\hfill \\ \hfill & {\widehat{w}}_s^{+},{\widehat{w}}_i^{-}\ge 0,s\in \mathbb{O},i\in \mathbb{I}.\hfill \end{array}$$

(8)

The optimal solution of (6) is obtained by

$$\begin{array}{cc}\hfill & w_i^{-\ast }={\widehat{w}}_i^{-\ast }/t^{\ast },i\in \mathbb{I},\hfill \\ \hfill & w_s^{+\ast }={\widehat{w}}_s^{+\ast }/t^{\ast },s\in \mathbb{O},\hfill \\ \hfill & {\lambda }_j^{\ast }={\widehat{\lambda }}_j^{\ast }/t^{\ast },j=1,\dots ,n,j\ne k,\hfill \end{array}$$

(9)

where $\left\{{\widehat{\lambda }}_j^{\ast },{\widehat{w}}_i^{-\ast },{\widehat{w}}_r^{+\ast },t^{\ast }\right\}$ is optimal solution of (8).

In a similar way, problem (7) is reduced to the following linear programming problem:

$$\begin{array}{cc}\hfill min& \left(t-\sum _{i\in \mathbb{I}}{\mu }_i{\widehat{s}}_i^{-}/R_i^{-}\right)\hfill \\ \hfill s.t.& t{x}_{ik}=\sum _{j=1,j\ne k}^n{x}_{ij}{\widehat{\lambda }}_j-t{w}_i^{-\ast }+{\widehat{s}}_i^{-},i\in \mathbb{I},\hfill \\ \hfill & t{y}_{ik}=\sum _{j=1,j\ne k}^n{y}_{ij}{\widehat{\lambda }}_j+t{w}_i^{+\ast }-{\widehat{s}}_i^{+},i\in \mathbb{O}.\hfill \\ \hfill & t+\sum _{i\in \mathbb{O}}{\nu }_i{\widehat{s}}_i^{+}/R_i^{+}=1,\hfill \\ \hfill & \sum _{j=1,j\ne k}^n{\widehat{\lambda }}_j=t,\hfill \\ \hfill & {\widehat{\lambda }}_j\ge 0,j=1,\dots ,n,j\ne k,\hfill \\ \hfill & {\widehat{s}}_i^{-}\ge 0,i\in \mathbb{I},\hfill \\ \hfill & {\widehat{s}}_i^{+}\ge 0,i\in \mathrm{O}.\hfill \end{array}$$

(10)

From the solution of (10), possible input excesses $s_i^{-\ast }$ and output shortfalls $s_i^{+\ast }$ can be determined as follows:

$$\begin{array}{cc}\hfill & s_i^{-\ast }={\widehat{s}}_i^{-\ast }/t^{\ast },i\in \mathbb{I},\hfill \\ \hfill & s_i^{+\ast }={\widehat{s}}_i^{+\ast }/t^{\ast },i\in \mathbb{O}.\hfill \end{array}$$

(11)

3.4. Weight Restrictions Adjustment

The weight restrictions in the CCR-AR model are specified exogenously. Moreover, there is no universal rule on how to choose them in a particular case. For each model, the weight restrictions are determined individually depending on the available data and the DM’s opinion. Therefore, to adjust the model, we recommend following the procedure described below (Listing 1).

Listing 1. Evaluation of regional environmental management systems.

Input: Inputs and outputs for all DMUs

Output: Efficiency scores

1: procedureAdjustWeightRestrictions

2:  Set initial weight restrictions and solve CCR-AR model (4) and (5)

3:  if DM does not agree with efficiency scores of some DMUs then

4:    do

5:     Construct sections of the frontier for these DMUs

6:     Identify the reasons for inconsistencies

7:     Determine new weights restrictions and solve CCR-AR model

8:    while The efficiency scores are not following the DM’s opinion

9:  end if

10: end procedure

Listing 1. Evaluation of regional environmental management systems.

Input: Inputs and outputs for all DMUs

Output: Efficiency scores

1: procedureAdjustWeightRestrictions

2:  Set initial weight restrictions and solve CCR-AR model (4) and (5)

3:  if DM does not agree with efficiency scores of some DMUs then

4:    do

5:     Construct sections of the frontier for these DMUs

6:     Identify the reasons for inconsistencies

7:     Determine new weights restrictions and solve CCR-AR model

8:    while The efficiency scores are not following the DM’s opinion

9:  end if

10: end procedure

Therefore, the process of decision-making for efficiency evaluation in regional environmental management systems can be described as follows.

First, we choose the CCR-AR model and set initial weights restrictions. If DM has no initial preferences, he/she can make them large enough so that they do not have a significant impact. At the second step, efficiency scores are calculated.

If DM disagrees with some of the obtained efficiency scores, then we construct two- and/or three-dimensional sections of the production possibility set in order to reveal the reasons for such inconsistencies. Algorithms for constructing such sections are given in [86,87]. The graphical representation of frontier sections provides a visual way to study a multidimensional production possibility set, which increases the intuition of the decision-maker, helps to produce more effective decisions, and reduces the decision-making time.

After the weights restrictions are corrected, the model is calculated again and DM checks new efficiency scores. If obtained efficiency scores are not in question, then the procedure stops; otherwise, the steps of correcting the weights are repeated.

Note that MCDM methods may be used for weights adjustment. This question requires a deep study and is not the purpose of this paper. Therefore, we consider it as a future research direction. The procedure describes the process of weights adjustment using a single DM’s opinion; for multiple DMs, it is necessary to use special methods of group decision-making [88,89].

In SBM model (8), DM can adjust weights ${\mu }_i$ and ${\nu }_s$. Therefore, in order to fit the model to the DM’s opinion, the procedure for SBM model can be written in a similar way and thus omitted.

4. Empirical Analysis

4.1. Data Source and Variables Selection

The data for the efficiency evaluation of regional environmental management systems were collected from the Russian Federal State Statistics Service’s regional statistical data [90,91]. The dataset includes all regions of Russia, with the exception of those in which the current costs of environmental protection measures are below the minimum of maintaining the operability of the regional environmental management system (10 million rubles were taken for the minimum threshold level). The efficiency scores of regional environmental management systems were calculated over the period from 2011 to 2017 in two-year steps, i.e., in 2011, 2013, 2015, and 2017 for three models: the regular CCR model with no weighting and scale shift restrictions (CCR), the CCR model with interval weighting and scale shift restrictions (CCR-AR), and the SBM model.

According to the Russian Federal State Statistics Service, the investments in fixed assets aimed at environmental protection and rational use of natural resources include the cost of construction, reconstruction (including the expansion and modernization) of facilities, the purchase of machinery, equipment, and vehicles. The current expenses on environmental protection include the costs of following activities: the maintenance and operation of fixed assets for environmental protection; measures for the preservation and recovery of the natural environment; the treatment of production and consumption wastes; the organization of control over emissions (discharges), production, and consumption wastes; environmental research; and environmental education.

As inputs, we considered the share of investments in fixed assets aimed at reducing environmental pollution ($x_1$) and the share of current costs in the region’s gross regional product (GRP) ($x_2$). As outputs, we consider indicators of a decrease in the specific level of pollution to the atmosphere, soil, and water, i.e., the difference in specific pollution before ($t_1$) the investments of both types (in fixed capital and current expenses) and after ($t_2$).

$$\begin{array}{cc}\hfill & y_{1,k}={\Delta }_k^{EP}=\frac{E{P}_{t_1,k}}{GR{P}_{t_1,k}}-\frac{E{P}_{t_2,k}}{GD{P}_{t_2,k}};\hfill \\ \hfill & y_{2,k}={\Delta }_k^{ET}=\frac{E{T}_{t_1,k}}{GR{P}_{t_1,k}}-\frac{E{T}_{t_2,k}}{GD{P}_{t_2,k}};\hfill \\ \hfill & y_{3,k}={\Delta }_k^{SW}=\frac{S{W}_{t_1,k}}{GR{P}_{t_1,k}}-\frac{S{W}_{t_2,k}}{GD{P}_{t_2,k}};\hfill \\ \hfill & y_{4,k}={\Delta }_k^W=\frac{W_{t_1,k}}{GR{P}_{t_1,k}}-\frac{W_{t_2,k}}{GD{P}_{t_2,k}};\hfill \\ \hfill & y_{5,k}={\Delta }_k^{WT}=\frac{W{T}_{t_1,i}}{GR{P}_{t_1,i}}-\frac{W{T}_{t_2,k}}{GR{P}_{t_2,k}};\hfill \\ \hfill & y_{6,k}={\Delta }_k^R=R_{t_2,k}-R_{t_1,k},\hfill \end{array}$$

(12)

where $EP$ is an emissions to the atmosphere from stationary sources (thousands tones), $GRP$ is a gross regional product in 2011 prices (millions rubles), $ET$ is an emissions to the atmosphere from road transport (thousands tones), $SW$ is a discharge or untreated sewage (millions m${}^3$), W is a fresh water consumption (millions m${}^3$), $WT$ is a waste generation (tones), R is a share of recycling and reuse of waste (%), and k is an index of the region. Max, min, and mean statistics of variables are shown in

Table 2. Dataset statistics.

Variables	Min	Max	Mean	St.Dev.
Inputs
$x_1$	−0.0	2.04	−0.25	0.30
$x_2$	−$1.44\times {10}^{-3}$	3.25	−0.50	0.37
Outputs
$y_1$	$-3.69\times {10}^{-4}$	$6.64\times {10}^{-4}$	−$2.51\times {10}^{-5}$	$1.00\times {10}^{-4}$
$y_2$	$-6.00\times {10}^{-4}$	$1.12\times {10}^{-3}$	−$6.50\times {10}^{-6}$	$1.04\times {10}^{-4}$
$y_3$	$-5.04\times {10}^{-4}$	$5.79\times {10}^{-4}$	−$3.90\times {10}^{-5}$	$9.34\times {10}^{-5}$
$y_4$	$-2.34\times {10}^{-3}$	$3.22\times {10}^{-3}$	−$1.59\times {10}^{-4}$	$4.88\times {10}^{-4}$
$y_5$	$-1690.76$	643.24	$-15.00$	138.11
$y_6$	$-0.904$	0.946	−0.0325	0.243

.

4.2. Weights Selection

We set the weight restrictions in the form of the ratio of the input/output weight coefficient to the virtual input/output. Such a model is a generalization of the assurance regions method (denoted in the work as CCR-AR). In this case, the upper and lower boundaries of the specified constraints show the relative significance of each indicator in relation to the others. In the CCR-AR model, for working with negative outputs, we perform a shift of the scale of indicators. For each indicator having negative values, the shift value was selected in such a way that all values were strictly positive.

As the second way of working with negative outputs in this paper, we used the non-radial SBM model. This model is quite general; it takes into account the presence of negative data, allows one to set weight coefficients for each indicator, and makes it possible to rank effective objects using the super-efficiency method.

As discussed earlier, many outputs have negative values, so in order to apply a model with restrictions on virtual weight coefficients (the CCR-AR model), it is necessary to perform a shift transformation of the corresponding variables. For each indicator that has negative values, the shift value was selected in such a way that all values were strictly positive.

After performing several steps of adjusting weights, the following estimates were obtained. The weight restrictions for the outputs in the CCR-AR model are shown in

Table 3. Weights in CCR-AR and SBM models.

Variable	Upper/Lower Bound of the Weight Restrictions in CCR-AR Model	Weight in SBM Model
$x_1$—share of investments in fixed assets aimed at reducing environmental pollution	—	0.5
$x_2$—share of current costs in the region’s GRP	—	0.5
$y_1$—difference in the intensity of emissions from stationary sources	0.1/0.3	0.182
$y_2$—difference in the intensity of emissions from road transport	0.1/0.3	0.182
$y_3$—difference in the intensity of discharge of untreated sewage	0.1/0.3	0.182
$y_4$—difference in the intensity of fresh water consumption	0.05/0.1	0.09
$y_5$—difference in the intensity of waste generation	0.1/0.3	0.182
$y_6$—difference in the share of recycling and reuse of waste	0.1/0.3	0.182

(second column); they are applied for all the DMUs. The values of the weights for the inputs and outputs of the SBM model we used in our computations are shown in Table 3 (third column).

4.3. DEA Evaluation Results

According to determined weights, CCR-AR and SBM models were applied to the dataset, and efficiency scores were evaluated. Comparing the results obtained, note that the traditional CCR model without weight restrictions and the CCR-AR model provides very close efficiency scores (

Table 4. Correlation of the efficiency scores in CCR, CCR-AR, and SBM models.

	CCR Model	CCR-AR Model	SBM Model
CCR model	1
CCR-AR model	0.9852815	1
SBM model	0.6622165	0.6295127	1

). Although the results of the SBM model correlate with the results of calculations based on the CCR model and the CCR-AR model, the value of the correlation coefficient is significantly lower than in the previous case.

A detailed analysis of the results obtained allows us to conclude that the CCR and CCR-AR models are more sensitive to the model inputs; according to these models, the regions with the lowest environmental costs are efficient. Low or zero input makes the region efficient even if some outputs have negative values. For example, in the Bryansk Oblast in 2011, the values of investments in fixed assets are zero. The result of environmental protection activities in 2011 was a slight improvement in all environmental indicators, except for the intensity of waste generation. Nevertheless, the environmental management system of the region is recognized as efficient according to the CCR model and the CCR-AR model, see

Table 5. Regions with the highest and lowest values of the efficiency score (comparison of 3 models).

CCR Model	CCR-AR Model	SBM Model
Regions with the highest values of the efficiency score
Bryansk Oblast-11	Moscow-11	Stavropol Krai-11
Moscow-11	Dagestan-11	Astrakhan Oblast-13
Dagestan-11	Dagestan-13	Arkhangelsk Oblast-11
Dagestan-13	Tuva Republic-15	Orenburg Oblast-13
Ivanovo Oblast-15	Bryansk Oblast-11	North Ossetia-Alania-13
Tuva Republic-15	Ivanovo Oblast-15	Dagestan-11
Ivanovo Oblast-17	Ivanovo Oblast-17	Tuva Republic-17
Novosibirsk Oblast-17	Moscow-13	Moscow Oblast-11
Moscow-13	Adygea-15	Kabardino-Balkar Republic-11
North Ossetia-Alania-13	Tuva Republic-13	Chuvash Republic-11
Adygea-15	Moscow-15	Kemerovo Oblast-15
Kostroma Oblast-15	Novosibirsk Oblast-17	Kostroma Oblast-15
Tuva Republic-13	Moscow-17	Kemerovo Oblast-17
Moscow-15	Mari El Republic-15	Karelia-11
Kabardino-Balkar Republic-11	Tuva Republic-17	Ivanovo Oblast-15
Regions with the lowest values of the efficiency score
Murmansk Oblast-15	Astrakhan Oblast-13	Murmansk Oblast-13
Murmansk Oblast-13	Murmansk Oblast-15	Kemerovo Oblast-13
Bashkortostan-11	Vologda Oblast-13	Krasnoyarsk Krai-13
Vologda Oblast-13	Sakha (Yakutia) Republic-17	Tver Oblast-11
Sakha (Yakutia) Republic-17	Bashkortostan-13	Sakha (Yakutia) Republic-17
Kemerovo Oblast-13	Murmansk Oblast-13	Sakha (Yakutia) Republic-13
Bashkortostan-13	Sakha (Yakutia) Republic-13	Krasnoyarsk Krai-15
Krasnoyarsk Krai-17	Krasnoyarsk Krai-17	Vologda Oblast-13
Sakha (Yakutia) Republic-13	Murmansk Oblast-17	Perm Krai-15
Murmansk Oblast-17	Kemerovo Oblast-13	Volgograd Oblast-15
Krasnoyarsk Krai-11	Krasnoyarsk Krai-11	Komi Republic-15
Krasnoyarsk Krai-15	Krasnoyarsk Krai-15	Khabarovsk Krai-11
Krasnoyarsk Krai-13	Krasnoyarsk Krai-13	Komi Republic-13
Volgograd Oblast-15	Volgograd Oblast-15	Amur Oblast-13
Lipetsk Oblast-11	Lipetsk Oblast-11	Lipetsk Oblast-11

. Similarly, the environmental management system of Moscow in 2011 was efficient, although the indicators of the intensity of waste generation in the region also deteriorated over the observed period. However, the region is efficient due to the low share of costs (investment in fixed assets is 0.06% of GRP, current costs also 0.06% of GRP). At the same time, the Lipetsk Oblast, which also has the only negative indicator of the intensity of waste generation, has the lowest efficiency, as the indicators of the region’s costs for environmental protection measures are among the highest: investments in fixed assets are 1.37% of GRP, and current costs are 3.25% of GRP.

The SBM model demonstrates a higher sensitivity to outputs: for example, the Stavropol Krai and the Arkhangelsk Oblast in 2011, as regions with the most effective environmental management systems, actually have all positive outputs. However, the regions following them in the performance rating already have one, two, or even three negative outputs, so the interpretation of the results of this model is controversial.

The spatial distribution of efficiency scores for CCR-AR and SBM models are showed in Figure 1 and Figure 2, respectively. Table 4 shows that the conventional CCR efficiency scores are very close to the CCR-AR model, and thus spatial distribution diagrams for the CCR model are not included in the paper. Although the results of the SBM model correlate with the results of calculations using the CCR model and the CCR-AR model, the value of the correlation coefficient is significantly lower than in the previous case.

Looking at the results as a whole, we can conclude that the efficiency of regional environmental management has deteriorated during the period under the study. Most likely, this is due to the deterioration of the economic situation in Russia in the period after 2014 due to the imposition of sanctions: incomplete utilization of production facilities, lack of funds for the introduction of more resource-efficient technologies, etc.

The highest potential for improvement in order to achieve efficiency in environmental management have two largest Russian regions in Eastern Siberia: Krasnoyarsk Krai and Yakutia. Therefore, special attention should be paid to the study of problems in the environmental management system in these regions. The regions that demonstrate the highest indicators of the effectiveness of environmental management systems (Tuva Republic, Ivanovo Oblast, and Moscow) are also worthy of closer study in order to identify and analyze the best practices and technologies of environmental management.

A more detailed analysis of the results allows us to conclude that the CCR and CCR-AR models are more sensitive to model inputs: according to these models, regions with the lowest environmental protection costs are effective. Low or zero cost values of at least one type immediately make the region efficient, even if some of the model outputs have negative values. The SBM model demonstrates a higher sensitivity to outputs, for example, regions, which are recognized as efficient according to the results of calculations, rarely have all outputs positive. However, the regions following them in efficiency ratings already have one, two, and even three negative outputs, so an interpretation of the results of this model is controversial. According to the findings above, some policy implications may be proposed for the evaluation of efficiency in regional environmental management systems in Russia.

5. Discussion and Conclusions

5.1. Discussion and Future Research Directions

In many previous studies, various DEA models are used in order to evaluate environmental efficiency. However, the efficiency evaluation of regional environmental management systems considered in this paper differs from other DEA applications because it does not use absolute variables, but changes over a certain period. As these changes may be not positive due to environmental degradation, negative values of variables are a common occurrence. At the same time, traditional DEA models have certain difficulties with negative data. To apply them, it is necessary to make some data transformations, which would inevitably be subjective. As a result, efficiency scores would be biased and decisions based on the use of such estimates would be unsubstantiated and possibly flawed. Therefore, from a mathematical point of view, the SBM models are more attractive for environmental efficiency problems with negative data. To perform the calculations, we used a fairly new model of Lin et al., which allows us to calculate a non-radial measure and at the same time correctly processes negative variables. In addition, the model used allows us to rank efficient units by calculating the super-efficiency score. It also has some attractive properties—it is monotonous, unit-invariant, and translation-invariant for both inputs and outputs.

The limitations of this study are as follows: (1) the choice of inputs and outputs of the model was limited due to the peculiarities of the statistical accounting of various aspects of environmental management in Russia. For example, Russian statistics do not separate greenhouse gas emissions, but consider them in one indicator along with all other emissions of gaseous pollutants. Therefore, it is not possible to analyze the effectiveness of regional environmental management in such an important area as combating climate change. (2) For the same reason (features of statistical accounting), we cannot compare the effectiveness of environmental management in Russian regions with other countries, for example, with the EU countries, although such comparisons are of significant practical interest. (3) As the study was limited to CRS models, it did not provide the information to analyze the difference in the technical efficiency and scale efficiency of regional systems of environmental management, which also is an important topic for decision-making. Despite its limitations, the study certainly adds to our understanding of the problems with inefficient investments in environmental protection measures in Russian regions.

A natural progression of this work is to analyze the efficiency of regional environmental management taking into account the peculiarities of ecological problems of each region. It can be done by assigning different weight restrictions to the outputs depending on whether the improvement of the ecological situation in this area is a priority for the region or not. Another direction of future research is might be an introduction of an artificial object into the sample, which demonstrates the highest possible investment efficiency at the current level of technological development. The introduction of such an artificial object changes the efficiency frontier and makes it possible to determine not only the efficiency of regions in relation to each other, but also their efficiency in comparison with those regions that use the best available technologies in all areas of environmental management.

5.2. Conclusions

Although the primary applicable result of this research is a rating of Russian regions, this result differs significantly from other Russian ecologic ratings by its methodology. The existing ratings normally evaluate either the eco-efficiency of the regional economy (e.g., the ecological rating of the Analytical Credit Rating Agency [92]), or use the expert estimates of regional events that have any connection to ecology (for example, the national economic rating of the national “Green Patrol” organization [93]). Apart from regional ratings, there are also some ratings of Russian cities, which evaluate the work of municipal governments on improving waste management and planting of greenery (e.g., the combined rating of the political “All-Russia People’s Front” movement and the Ministry of Natural Resource and Ecology [94]). However, none of the methodologies used in these ratings account for the efficiency of spending on environmental protection. Thus, a direct comparison of our three DEA models with results of other ratings is impossible, and doing so will not allow determining which of the models better reflects reality. This could be considered a limitation of our research.

Nevertheless, accounting for the differences in calculations of efficiency measures in each of our models, we consider the SBM model to provide the most accurate results. First, this model does not allow for a situation where some ecologic effects have an overly large weight coefficient and some others have an overly small one. In other words, we cannot consider regions that had huge successes in one area of environmental protection and complete failures in other areas as efficient. Second, this model has a higher discriminating power, which allows using strict ranking.

As for practical application of our methodology for creating ratings of regional ecologic management efficiency, we hold a positive viewpoint, as all necessary indicators for rating calculation are provided by official statistical organizations and are openly available. Our rating is based on quantitative indicators, can be calculated independently of regional authorities, and excludes any subjective influences. As any other official rating, it could be used for stimulating regional authorities to take genuine action in the sphere of ecology. However, unlike other ratings, the DEA-SBM model may also stimulate an optimization of environmental protection spending and, therefore, an introduction of technological and organizational eco-innovations.

The results of DEA models provide a Big Picture in the field of environmental management at the regional level. Taking into account the extreme non-transparency of this area in Russia (for example, the absence of publicly available information on what the budgetary funds allocated for environmental protection were spent on), they allow any independent researcher to identify the most problematic regions. A further detailed analysis of the environmental management processes will be required in order to determine the true reasons of inefficiency; however, DEA modeling helps to point out the potential corruption or technological violations in the region.