A Hierarchical Aggregation Approach for Indicators Based on Data Envelopment Analysis and Analytic Hierarchy Process

Abstract: This research proposes a hierarchical aggregation approach using Data Envelopment Analysis (DEA) and Analytic Hierarchy Process (AHP) for indicators. The core logic of the proposed approach is to reflect the hierarchical structures of indicators and their relative priorities in constructing composite indicators (CIs), simultaneously. Under hierarchical structures, the indicators of similar characteristics can be grouped into sub-categories and further into categories. According to this approach, we define a domain of composite losses, i.e., a reduction in CI values, based on two sets of weights. The first set represents the weights of indicators for each Decision Making Unit (DMU) with the minimal composite loss, and the second set represents the weights of indicators bounded by AHP with the maximal composite loss. Using a parametric distance model, we explore various ranking positions for DMUs while the indicator weights obtained from a three-level DEA-based CI model shift towards the corresponding weights bounded by AHP. An illustrative example of road safety performance indicators (SPIs) for a set of European countries highlights the usefulness of the proposed approach.


Introduction
Individual indicators are multidimensional measures that can assess the relative positions of entities (e.g., countries) in a given area [1].A Composite indicator (CI) is a mathematical aggregation of individual indicators into a single score.Two simple but popular aggregation methods in the context of multi-criteria decision-making (MCDM) are the weighted sum (WS) method and the weighted product (WP) method [2].Some researchers have recently pointed out that the WP method may have some advantages over the WS method in CI construction [3][4][5].However, the assignment of weights to indicators is still a main source of difficulty in the application of these methods.Fortunately, the recent methodological advances in operations research and management science (OR/MS) have provided us with two powerful tools, namely data envelopment analysis (DEA) and analytic hierarchy process (AHP), which can be used as weighting and aggregation tools in CI construction.
Data Envelopment Analysis is a nonparametric method to assess the relative efficiency of a group of DMUs based on their distance from the best-practice frontier.In this method each DMU can freely choose its own weights to maximize its performance [6].
The standard DEA models are formulated using multiple inputs and multiple outputs of DMUs.The application of this group of models in CI construction can be found in [7][8][9].However, in recent years much more attention has been focused on the application of a new group of DEA models in the field of composite indicators which is known as the "benefit of the doubt" (BOD) approach.
In the BOD approach, all indicators are treated as outputs without explicit inputs, i.e., the property of "the larger is the better" [10][11][12].
In light of the possibility of neglecting the priority of various indicators, some critics have questioned the validity and stability of CIs obtained via DEA.Decision makers (DMs), in some contexts, have value judgments concerning the relative priority of indicators that should be taken into account in CI construction.
Alternatively, AHP is a systematic MCDM method to generate the true or approximate weights based on the well-defined mathematical structures of pairwise comparison metrics.
The application of AHP in CI construction provides a priori information about the relative priority of indicators [13][14][15].AHP usually involves three basic functions: structuring complexities, measuring on a ratio-scale and synthesizing [16].One of the advantages of AHP is its high flexibility to be combined with the other OR/MS techniques [17].AHP can be combined with DEA in different ways.The most common approach is the estimation of parameters of weight restrictions on the DEA models.AHP estimates the appropriate values for the parameters in the absolute weight restrictions [18], relative weight restrictions [19][20][21][22][23], virtual weight restrictions [24,25] and restrictions on changes of input (output) units [26].
There are a number of other methods that do not necessarily apply additional restrictions to a DEA model.Such as converting the qualitative data in DEA to the quantitative data using AHP [27][28][29][30][31][32][33][34], ranking the efficient/inefficient units in DEA models using AHP in a two stage process [35][36][37], weighting the efficiency scores obtained from DEA using AHP [38], weighting the inputs and outputs in the DEA structure [39][40][41][42], constructing a convex combination of weights using AHP and DEA [43] and estimating missing data in DEA using AHP [44].
The recent studies by Pakkar [45][46][47][48][49][50] demonstrate the effects of imposing weight bounds on the different variants of DEA models using AHP.To this end, AHP has been applied in single-level DEA models [47][48][49][50] and two-level DEA models [45,46].Due to the complexity of the hierarchical structures of indicators, this paper applies AHP into an additive three-level DEA model in the context of CI construction.Theoretically, the approach proposed in this paper may also be considered as the additive form of the multiplicative three-level DEA-based CI approach to constructing CIs proposed by [51].In a three-level hierarchy, the indicators of similar characteristics can be grouped into sub-categories and further into categories.A three-level DEA model entirely reflects the characteristics of a generalized multiple level DEA model developed in [52,53].Since the proposed approach uses AHP in an additive three-level DEA-based model, it contributes to the set of methods currently available for CI construction.

Methodology
This research has been organized to proceed along the following stages (Figure 1): Computing the composite value of each DMU using one-level DEA-based CI model ( 4).
The computed composite values are applied in three-level DEA-based CI model (6).

2.
Computing the priority weights of indicators for all DMUs using AHP, which impose weight bounds into model (6).

3.
Obtaining an optimal set of weights for each DMU using three-level DEA-based CI model (6) (minimum composite loss η).

4.
Obtaining an optimal set of weights for each DMU using model (6) bounded by AHP (maximum composite loss κ).Note that if the AHP weights are added to model (6), we obtain model (10).

5.
Measuring the performance of each DMU in terms of the relative closeness to the priority weights of indicators.For this purpose, we develop parameter-distance model (11).Increasing a parameter in a defined range of composite loss we explore how much a DM can achieve its goals.This may result in various ranking positions for a DMU in comparison to the other DMUs.

DEA-Based CI Model
A DEA-based CI model can be formulated similar to a classical DEA model in which all data are treated as outputs without explicit inputs [54].In the following, and in line with the more common CI terminology, we will often refer to outputs as "indicators".In order to eliminate the scale differences between all (output) indicators, and moreover, to ensure that all of them are in the same direction of change the normalized counterparts of indicators, using the distance to reference method, are computed as follows [1]: ).

DEA-Based CI Model
A DEA-based CI model can be formulated similar to a classical DEA model in which all data are treated as outputs without explicit inputs [54].In the following, and in line with the more common CI terminology, we will often refer to outputs as "indicators".In order to eliminate the scale differences between all (output) indicators, and moreover, to ensure that all of them are in the same direction of change the normalized counterparts of indicators, using the distance to reference method, are computed as follows [1]: , ŷrpmaxq " max t ŷr1 , ŷr2 , ..., ŷrn u for desirable indicators y rj " ŷrpminq ŷrj , ŷrpminq " min t ŷr1 , ŷr2 , ..., ŷrn u for undesirable indicators (2) where y rj is the normalized value of (output) indicator r (r " 1, 2.., s) for DMU j (j = 1, 2, . . ., n).Now assume that all DMUs have unit input i (i " 1, 2, ..., m).Then the fractional CCR-DEA model can be developed as follows [55]: where CI k is the composite indicator of DMU under assessment.k is the index for the DMU under assessment where k ranges over 1, 2, . . ., n. v i and u r are the weights of input i (i " 1, 2, ..., m) and (output) indicator r (r " 1, 2.., s).The first set of constraints assures that if the computed weights are applied to a group of n DMUs, (j " 1, 2, ..., n), they do not attain a composite score of larger than 1.The second set of constraints indicates the non-negative conditions for the model variables.
Introducing the constraint m ř i"1 v i " 1 and performing the operation of substitution, an equivalent linear model can be formulated as follows: s ÿ r"1 u r y rj ď 1 @j u r ą 0 @r.
Model ( 4) looks like a DEA model without inputs that extends the standard DEA methodology to the field of CI construction.

Three-Level DEA-Based CI Model
We develop a three-level DEA model to aggregate the performance of indicators under the (sub) category they belong to by a weighted-average method (Figure 2).Let y ll 1 rj be the value of indicator r (r " 1, 2, ..., s) of sub-category l 1 pl 1 " 1, 2, ..., S 1 q of category l pl " 1, 2, ..., Sq for DMU j (j " 1, 2, ..., n) after normalizing the original data.Let u ll 1 r be the internal weight of indicator r of sub-category l 1 of category l while s ř r"1 u ll 1 r " 1.Then the value of sub-category l 1 of category l for the DMU j is defined as y ll 1 j " s ř r"1 u ll 1 r y ll 1 rj .Let p ll 1 be the internal weight of sub-category l 1 of category l while Then the value of category l is defined as y lj " p ll 1 y ll 1 j .Let p l be the weight of category l.To develop a linear model, the new multiplier of indicator r of sub-category l 1 of category l is defined as: u 1 ll 1 r " p l p ll 1 u ll 1 r .Similarly, the new multiplier of sub-category l 1 of category l is defined as: p 1 ll 1 " p l p ll 1 .Consequently, a linear three-level DEA model for indicators can be developed as follows: We develop our formulation based on the generalized distance model [56,57] in such a way that the hierarchical structures of indicators, using a weighted-average approach, are taken into consideration [52,53].Let ) be the best attainable composite value for the DMU under assessment, calculated from model ( 4).We want the composite value () where t D is a distance measure and t represents the distance parameter.Our definition of "closest" is that the largest distance is at its minimum.On the other hand, the largest distance completely dominates when t .For t ,

 
Model (6) identifies the minimum composite loss  (eta) needed to arrive at an optimal set of weights.The first constraint ensures that each DMU loses no more than  of its best attainable We develop our formulation based on the generalized distance model [56,57] in such a way that the hierarchical structures of indicators, using a weighted-average approach, are taken into consideration [52,53].Let CI k (k " 1, 2, ..., n) be the best attainable composite value for the DMU under assessment, calculated from model (4).We want the composite value CI k pu 1 ll 1 r q, calculated from the set of weights u 1 ll 1 r , to be closest to CI k .The degree of closeness between CI k pu 1 ll 1 r q and CI k is measured as D t " rCI k ´CI k pu 1 ll 1 r q t s 1{t with t ě 1, where D t is a distance measure and t represents the distance parameter.Our definition of "closest" is that the largest distance is at its minimum.On the other hand, the largest distance completely dominates when t " 8.For t " 8, the distance measure is reduced to D 8 " max Model (6) identifies the minimum composite loss η (eta) needed to arrive at an optimal set of weights.The first constraint ensures that each DMU loses no more than η of its best attainable composite value, CI k .The second set of constraints satisfies that the composite values of all DMUs are less than or equal to their upper bound of CI j .
Two sets of constraints are added to model ( 6): are indicator multipliers.This implies that the sum of weights under each (sub-) sub-category equals to the weight of that (sub-) sub-category.It should be noted that the original (or internal) weights used for calculating the weighted averages are obtained as u ll 1 r " u 1 ll 1 r {p 1 ll 1 and p ll 1 " p 1 ll 1 {p l .

Prioritizing Indicator Weights Using AHP
Model (6) identifies the minimum composite loss η (eta) needed to arrive at a set of weights of indicators by the internal mechanism of DEA.On the other hand, the priority weights of indicators, and the corresponding (sub) categories are defined out of the internal mechanism of DEA by AHP (Figure 3).In order to more clearly demonstrate how AHP is integrated into the three-level DEA-based CI model, this research presents an analytical process in which indicator weights are bounded by the AHP method.The AHP procedure for imposing weight bounds may be broken down into the following steps: Step 1: A decision maker makes a pairwise comparison matrix of different criteria, denoted by A , with the entries of lq a ( 1, 2,..., ) l q S  .The comparative importance of criteria is provided by the decision maker using a rating scale.Saaty [16] recommends using a 1-9 scale.
Step 2: The AHP method obtains the priority weights of criteria by computing the eigenvector of matrix A (Equation ( 7)), To determine whether or not the inconsistency in a comparison matrix is reasonable the random consistency ratio, .. CR, can be computed by the following equation: .
where .. RI is the average random consistency index and N is the size of a comparison matrix.In a similar way, the priority weights of (sub-) sub-criteria under each (sub-) criterion can be computed.
To obtain the weight bounds for indicator weights in the three-level DEA-based CI model, this study aggregates the priority weights of three different levels in AHP as follows: In order to more clearly demonstrate how AHP is integrated into the three-level DEA-based CI model, this research presents an analytical process in which indicator weights are bounded by the AHP method.The AHP procedure for imposing weight bounds may be broken down into the following steps: Step 1: A decision maker makes a pairwise comparison matrix of different criteria, denoted by A, with the entries of a lq pl " q " 1, 2, ..., Sq.The comparative importance of criteria is provided by the decision maker using a rating scale.Saaty [16] recommends using a 1-9 scale.
Step 2: The AHP method obtains the priority weights of criteria by computing the eigenvector of matrix A (Equation ( 7)), w " pw 1 , w 2 , ..., w S q T , which is related to the largest eigenvalue, λ max .
Aw " λ max w To determine whether or not the inconsistency in a comparison matrix is reasonable the random consistency ratio, C.R., can be computed by the following equation: where R.I. is the average random consistency index and N is the size of a comparison matrix.In a similar way, the priority weights of (sub-) sub-criteria under each (sub-) criterion can be computed.
To obtain the weight bounds for indicator weights in the three-level DEA-based CI model, this study aggregates the priority weights of three different levels in AHP as follows: e ll 1 " 1 and where w l is the priority weight of criterion l (l " 1, ..., S) in AHP, e ll 1 is the priority weight of sub-criterion l 1 pl 1 " 1, 2, ..., S 1 q under criterion l and f ll 1 r is sub-sub-criterion r (r " 1, ..., s) under sub-criterion l 1 .In order to estimate the maximum composite loss κ (kappa) necessary to achieve the priority weights of indicators for each DMU the following linear program is proposed: Min κ (10) The first sets of constraints change the AHP computed weights to weights for the new system by means of a scaling factor α. The scaling factor α is added to avoid the possibility of contradicting constraints leading to infeasibility or underestimating the relative composite scores of DMUs [58].The optimal solution to model (10) produces a set of weights for indicators that are used to compute the performance of DMUs.
It should be noted that incorporating absolute weight bounds, using AHP, for indicator weights in a DEA-based CI model is consistent with the common practice of constructing composite indicators.According to this practice, the priority weights of indicators can be used directly in an aggregation function to synthetize indicators' values into composite values [1].In addition, this form of placing restrictions on indicator weights simply allows us to identify a specific range of variation between two systems of weights obtained from models ( 6) and (10).

A Parametric Distance Model
We can now develop a parametric distance model for various discrete values of parameter θ such that η ď θ ď κ.Let u 1 ll 1 r pθq be the weights of indicators for a given value of parameter θ, where indicators are under sub-category l 1 pl 1 " 1, 2, ..., S 1 q of category l (l " 1, 2, ..., S).Let u 1 ll 1 r pκq be the priority weights of indicators under sub-category l 1 of category l, obtained from model (10).
Our objective is to minimize the total deviations between u 1 ll 1 r pθq and u 1 ll 1 r pκq with the shortest Euclidian distance measure subject to the following constraints: Because the range of deviations computed by the objective function is different for each DMU, it is necessary to normalize it by using relative deviations rather than absolute ones [59].Hence, the normalized deviations can be computed by: where Z k pθq is the optimal value of the objective function for η ď θ ď κ.We define ∆ k pθq as a measure of closeness which represents the relative closeness of each DMU to the weights obtained from model (10) in the range [0, 1].Increasing the parameter pθq, we improve the deviations between the two systems of weights obtained from models (6) and (10) which may lead to different ranking positions for each DMU in comparison to the other DMUs.It should be noted that in a special case where the parameter θ " κ " 0, we assume ∆ k pθq = 1.

A Numerical Example: Road Safety Performance Indicators
In this section we present the application of the proposed approach to assess the road safety performance of a set of 13 European countries (or DMUs): Austria (AUT), Belgium (BEL), Finland (FIN), France (FRA), Hungary (HUN), Ireland (IRL), Lithuania (LTU), Netherlands (NLD), Poland (POL), Portugal (PRT), Slovenia (SVN), Sweden (SWE) and Switzerland (CHE).The data for eleven hierarchical indicators that compose SPIs for these countries have been adopted from [52].The eight SPIs related to alcohol and speed are undesirable indicators while the three SPIs related to protective systems are desirable ones.The resulting normalized data based on Equations ( 1) and ( 2) are presented in Table 1.
Taking the percentage of speed limit violation on rural roads as an example, Slovenia performs the best (1.000) while Poland the worst (0.014) and all other countries' values lie within this interval.
The results of the AHP model for prioritizing hierarchical SPIs as constructed by the author in Expert Choice software are presented in Table 2.One can argue that the priority weights of SPIs must be judged by road safety experts.However, since the aim of this section is just to show the application of the proposed approach on numerical data, we see no problem to use our judgment alone.Solving model (6) for the country under assessment, we obtain an optimal set of weights with minimum composite loss pηq.Since the raw data are normalized, the weights obtained from this model are meaningful and have an intuitive explanation.As a result, we can later set meaningful bounds on the weights in terms of the relative priority of indicators.Taking Austria as an example in Table 3, with the composite value of one obtained from model (4), we can observe that the alcohol-related fatality rate is 26 times less important than the mean speed of vehicles on motorways and about 3.2 times less important than the daytime usage rate of child restraints.Clearly, the other indicators are ignored in this assessment by assigning zero weights which are equivalent to excluding those indicators from the analysis.This kind of situation can be remedied by including the opinion of experts in defining the relative priority of indicators.It should be noted that the composite value of all countries calculated from model ( 6) is identical to that calculated from model (4).Therefore, the minimum composite loss for the country under assessment is η " 0 (Table 4).This implies that the measure of relative closeness to the AHP weights for the country under assessment is ∆ k pηq " 0. On the other hand, solving model (10) for the country under assessment, we adjust the priority weights of hierarchical SPIs obtained from AHP in such a way that they become compatible with the weights' structure in the three level DEA-based CI models.Table 5 presents the optimal weights of hierarchical SPIs as well as its scaling factor for all countries.Note that the priority weights of AHP used for incorporating weight bounds on indicator weights in model (10) are obtained as u ll 1 r " u 1 ll 1 r α .Similarly, the priority weights of AHP at criteria level can be obtained as w l " p l α while In addition, The priority weights of AHP at sub-criteria and sub-sub-criteria levels can be obtained as e ll 1 " p 1 ll 1 {p l and f ll 1 r " u 1 ll 1 r {p 1 ll 1 , respectively.The maximum composite loss for each country to achieve the corresponding weights in model ( 10) is equal to κ (Table 4).As a result, the measure of relative closeness to the priority weights of SPIs for the country under assessment is ∆ k pκq = 1.Going one step further to the solution process of the parametric distance model (11), we proceed to the estimation of total deviations from the AHP weights for each country while the parameter θ is 0 ď θ ď κ.Table 6 represents the ranking position of each country based on the minimum deviation from the priority weights of indicators for θ " 0. It should be noted that in a special case where the parameter θ " κ " 0 we assume ∆ k pθq " 1. Table 6 shows that Switzerland (CHE) is the best performer in terms of the CI value and the relative closeness to the priority weights of indicators in comparison to the other countries.Nevertheless, increasing the value of θ from 0 to κ has two main effects on the performance of the other countries: improving the degree of deviations and reducing the value of composite indicator.This, of course, is a phenomenon, one expects to observe frequently.The graph of ∆pθq versus θ, as shown in Figure 4, is used to describe the relation between the relative closeness to the priority weights of indicators and composite loss for each country.This may result in different ranking positions for each country in comparison to the other countries (Appendix A).
Appendix A

Figure 1 .
Figure 1.A hierarchical aggregation approach for indicators using a three-level Data Envelopment Analysis (DEA) and Analytic Hierarchy Process (AHP).
is the composite indicator of DMU under assessment.k is the index for the DMU under assessment where k ranges over 1, 2,…, n .i v and r u are the weights of input i ( 1, 2,..., im  ) and (output) indicator r ( 1, 2.., rs  ).The first set of constraints assures that if the computed weights are applied to a group of n DMUs, ( 1, 2,..., jn  ), they do not attain a composite score of larger than 1.The second set of constraints indicates the non-negative conditions for the model variables.

Figure 1 .
Figure 1.A hierarchical aggregation approach for indicators using a three-level Data Envelopment Analysis (DEA) and Analytic Hierarchy Process (AHP).

Figure 2 .
Figure 2. A three-level DEA framework for hierarchical indicators.

Figure 2 .
Figure 2. A three-level DEA framework for hierarchical indicators.

3 .
. The second set of constraints satisfies that the composite values of all DMUs are less than or equal to their upper bound of multipliers.This implies that the sum of weights under each (sub-) sub-category equals to the weight of that (sub-) sub-category.It should be noted that the original (or internal) weights used for calculating the weighted averages are obtained as Prioritizing Indicator Weights Using AHP Model (6) identifies the minimum composite loss  (eta) needed to arrive at a set of weights of indicators by the internal mechanism of DEA.On the other hand, the priority weights of indicators, and the corresponding (sub) categories are defined out of the internal mechanism of DEA by AHP (Figure3).

Figure 3 .
Figure 3.The AHP model for prioritizing indicators.

Figure 3 .
Figure 3.The AHP model for prioritizing indicators.

Table 1 .
Normalized data on the eleven hierarchical safety performance indicators (SPIs).

Table 2 .
The AHP hierarchical model for SPIs.

Table 4 .
Minimum and maximum losses in composite values for each country.

Table 6 .
The ranking position of each country based on the minimum distance to priority weights of SPIs.

Table A1 .
The measure of relative closeness to the priority weights of hierarchical SPIs [∆ k pθq ] vs. composite loss [θ] for each country.