Next Article in Journal
Mutual-Inductance Computation between Coaxial Thin Conical Sheet Inductor and the Circular Loop in Air
Next Article in Special Issue
Decision-Making Model of Production Data Management for Multi-Quality Characteristic Products in Consideration of Industry 4.0
Previous Article in Journal
Magnetic Properties of Gd-Doped Fe3O4 Nanoparticles
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Robust Additive Value-Based Efficiency Analysis with a Hierarchical Structure of Inputs and Outputs

by
Anna Labijak-Kowalska
,
Miłosz Kadziński
* and
Weronika Mrozek
Faculty of Computing and Telecommunications, Poznan University of Technology, Piotrowo 2, 60-965 Poznań, Poland
*
Author to whom correspondence should be addressed.
Appl. Sci. 2023, 13(11), 6406; https://doi.org/10.3390/app13116406
Submission received: 25 April 2023 / Revised: 15 May 2023 / Accepted: 23 May 2023 / Published: 24 May 2023
(This article belongs to the Special Issue Decision Support Systems: Novel Applications and Future Perspectives)

Abstract

:
We introduce a novel methodological framework based on additive value-based efficiency analysis. It considers inputs and outputs organized in a hierarchical structure. Such an approach allows us to decompose the problem into manageable pieces and determine the analyzed units’ strengths and weaknesses. We provide robust outcomes by analyzing all feasible weight vectors at different hierarchy levels. The analysis concerns three complementary points of view: distances to the efficient unit, ranks, and pairwise preference relations. For each of them, we determine the exact extreme results and the distribution of probabilistic results. We apply the proposed method to a case study concerning the performance of healthcare systems in sixteen Polish voivodeships (provinces). We discuss the results based on the entire set of factors (the root of the hierarchy) and three subcategories. They concern health improvement of inhabitants, efficient financial management, and consumer satisfaction. Finally, we show the practical conclusions that can be derived from the hierarchical decomposition of the problem and robustness analysis.

1. Introduction

Data Envelopment Analysis (DEA) measures the relative efficiency of Decision Making Units (DMUs) that convert inputs to outputs. It was originally proposed by Charnes et al. [1] as a nonparametric approach, making no assumptions about the production frontier or the weights assigned to the various factors relevant to the analysis. To assess the efficiency of DMUs, they are compared to the best-practice frontier determined by the group of units with the most favorable input–output performance. The traditional methods divide the units into efficient ones, i.e., those on the efficient frontier, and inefficient ones, i.e., those below the frontier. Due to its versatility, DEA has been widely used in various areas such as management, economics, agriculture, education, healthcare, and logistics [2]. The recent example applications concerned the assessment of public administration [3] and the urban rail transit network [4].
Since its first formulation, DEA has been extended in multiple ways [5,6]. For example, various efficiency models have been introduced to admit static or dynamic analysis or handle constant or variable returns to scale. In particular, an additive model was formulated to guarantee that the units it indicates as efficient satisfy this property in Koopman’s sense [7]. However, this model has also been criticized for assuming equal weights of all factors, vulnerability to the factors’ scale differences, and nonintuitive interpretation of the efficiency scores. These drawbacks have motivated the development of an additive value-based efficiency analysis [8,9], inspired by Multi-Attribute Value Theory (MAVT) [10]. This model transforms the input and output values using the marginal functions. Such per-criterion components are aggregated into a comprehensive efficiency measure with an additive value model incorporating weights assigned to various factors. The units that attain the greatest comprehensive value for at least one feasible weight vector are deemed efficient. Such an analysis is insensitive to scale problems due to applying value functions with a common scale. Moreover, the efficiency scores have an intuitive interpretation built on the notion of “min–max regret”. Note that the hybrid methods combining ideas from DEA and Multiple Criteria Decision Analysis (MCDA) have become more and more popular in recent years (see, e.g., [11,12]).
This paper contributes to the literature concerning an additive value-based efficiency analysis in a three-fold way. This methodology handles only flat structures of inputs and outputs considered at the same level, without subcategories [8]. Hence, our first contribution consists of adjusting it to handle hierarchical structures of factors used to assess the performance of DMUs. This is useful in real-world decision analysis for a few reasons. First, it helps to structure inputs and outputs logically and systematically. The higher-level factors are more general, whereas those at lower hierarchy levels are more specific. Moreover, when new information becomes available, the hierarchy can be easily modified or updated, allowing it to handle evolving decision problems. Second, a hierarchical decomposition of factors allows for the breaking down of complex problems into manageable, coherent pieces representing different levels of abstraction. By analyzing the efficiency at various levels of the hierarchy, it is possible to understand the strengths and weaknesses of DMUs and explain the comprehensive results taking into account their evolution along the hierarchy. Third, a hierarchical structure of factors makes efficiency analysis more transparent, flexible, and adaptable. In particular, we support the trade-off analysis, where weights can be associated with lower and higher-level categories of factors, and hence, their relative and absolute impact can be controlled more easily. In this regard, we incorporate the preferences elicited at each hierarchy level into the analysis. These preferences form the linear weight restrictions between factor categories at the same level.
The benefits of using a hierarchical structure have been explored in MCDA. The example methods that handle such a decomposition include the Analytical Hierarchy Process (AHP) [13], the Multiple Criteria Hierarchy Process (MCHP) [14,15], and ELECTRE-III-H [16]. In the DEA context, the first attempt was made with a two-layer nonlinear model [17] and its linear counterpart [18]. Then, Ref. [19] proposed a multiple-layer DEA model (MLDEA) handling an arbitrary number of levels of inputs and outputs. Further, MLDEA was combined with AHP to consider relative priorities of various factors, mainly in the scenarios where DEA is used as a mathematical tool for constructing so-called composite indicators [20,21]. Finally, the latter approach was generalized to the setting of Network and Fuzzy DEA [22]. The above-mentioned DEA models require inputs and outputs to be considered in separate hierarchies. We fill this research gap by admitting a single multiple-layer hierarchical structure containing inputs and outputs. In this way, the properly defined efficiency can be analyzed in each hierarchy node.
Second, in the proposed framework, we go beyond classifying the DMUs only into efficient and inefficient, as in the original value-based efficiency analysis [8]. This is attained by verifying the robustness of efficiency results observable for the entire space of feasible input and output weights. We focus on three perspectives: distances to the efficient DMU, ranks, and pairwise preference relations. For each of them, we compute the exact (necessary, possible, and extreme) outcomes by solving dedicated mathematical programming models. Moreover, we estimate the distribution of results using Monte Carlo simulations. The proposed framework is inspired by [23], being adapted to the multiple-level hierarchy value-based efficiency analysis. The formulations of dedicated procedures and types of considered results are similar to those considered in [24]. However, we admit the stakeholders studying the stability of efficiency outcomes in each hierarchy node instead of forcing them to consider all inputs and outputs simultaneously. In this regard, the main challenge is adequately handling the indicator weights considered at different hierarchy levels. We also formulate the properties of the exact efficiency outcomes observed along the hierarchy tree. Typically, they relate the results observed in all children nodes of some more general category to the outcomes obtained in the parent node. Hence, they help understand the evolution of necessary, impossible, and extreme conclusions.
Third, we apply the proposed framework to a case study concerning healthcare. As noted in [25], the efficiency of using resources to ensure a decent level of healthcare has become one of the most critical public policy issues in recent decades. Its assessment can be conducted from the perspective of the entire system or individual organization (e.g., hospitals). A detailed review of the applications of efficiency analysis in healthcare can be found in [26], and a detailed description of the healthcare system in Poland is given in [25].
We analyzed nine indicators capturing the quality of healthcare systems in sixteen Polish voivodeships. The indicators were grouped under three main categories: inhabitants’ health improvement, financial management, and consumer satisfaction. We elicited the preferences in the form of marginal value functions for all inputs and outputs and weight constraints. We report the results given a comprehensive efficiency index encompassing all relevant viewpoints and the three subproblems that allow for an understanding of each voivodeship’s strong and weak points. The paper’s three significant contributions, along with their essential aspects, are summarized in Figure 1.
The paper’s remainder is organized as follows. Section 2 describes an additive value-based efficiency model. Section 3 defines a hierarchical structure of inputs and outputs, while Section 4 describes a respective framework for robustness analysis. In Section 5, we report the outcomes of a case study concerning the efficiency assessment of the healthcare system in Poland. The last section concludes the paper.

2. Additive Value-Based Efficiency Analysis

Let us consider a set of DMUs D = { D 1 , , D K } . In value-based analysis, efficiency E o of D M U o D is defined using an additive value model:
E o = q = 1 Q w q · u q ( D M U o ) ,
where w q is the weight of factor q (i.e., input x x = { x 1 , x 2 , , x m } or output y y = { y 1 , y 2 , , y n } ), such that q = 1 Q w q = 1 , and u q is a monotonic marginal value function for q. It is nondecreasing for outputs and nonincreasing for inputs. To verify if D M U o is efficient, we solve the following Linear Programming (LP) model, minimizing the maximal distance of efficiency of D M U o to any other unit:
Minimize d o
s.t.
q = 1 Q w q u q ( D M U k ) q = 1 Q w q u q ( D M U o ) d o ,   for   k = 1 , , K , d o 0 , q = 1 Q w q = 1 , w q 0 , q = 1 , , Q , w S w ,
where S w is the feasible weight vector space delimited by the linear constraints. The optimal value of d o , denoted by d o * , represents the minimal distance of D M U o from the efficient unit. If d o * = 0 , D M U o attains the greatest efficiency for at least one feasible weight vector, which implies that it is efficient. When d o * > 0 , for all feasible weights, there is at least one unit with an efficiency greater than E o , denoting the lack of efficiency of D M U o .

3. A Hierarchical Structure of Inputs and Outputs

The DMUs consume m inputs x and produce n outputs y . To simplify the notation, we aggregate all inputs and outputs into a single set of factors f = x y = { f 1 , f 2 , , f Q 0 } . Set f forms level 0 of the hierarchy. These factors are grouped into Q 1 categories of the first level, named C ( 1 ) = { c 1 ( 1 ) , c 2 ( 1 ) , , c Q 1 ( 1 ) } . Analogously, the first-level categories can be grouped into second-level categories, forming a set C ( 2 ) = { c 1 ( 2 ) , c 2 ( 2 ) , , c Q 2 ( 2 ) } , etc. The entire structure contains L levels. In the last (L-th) level, there is only a single category ( c 1 ( L ) ), called a root.
From the mathematical viewpoint, the factors and categories form a tree (see Figure 2). The set of all nodes in the tree (factors and categories) is denoted by N = f C ( 1 ) C ( 2 ) C ( L ) . For each node t N \ { r o o t } , we define its parent p ( t ) as a category in which it is directly contained. The set of direct children of category c i ( l ) is marked as c h ( c i ( l ) ) = { t N : p ( t ) = c i ( l ) } . The set of indirect children of category c i ( l ) contains all direct children of c i ( l ) ( c h ( c i ( l ) ) ) and their direct and indirect children until reaching the tree’s leaves. For each category at hierarchy level c i ( l ) , we define set A c i ( l ) as a subset of f (inputs and outputs), which are the indirect children of c i ( l ) . In particular, all factors are indirect children of the root category, i.e., A c 1 ( L ) = f . On the contrary, for an elementary factor f, A f is a singleton, i.e., A f = { f } , f f . To maintain the spirit of DEA, for each category c i ( l ) , the respective set of factors ( A c i ( l ) ) needs to contain at least one input and one output, i.e., A c i ( l ) x and A c i ( l ) y , for l = 1 , 2 , , L , i = 1 , 2 , , Q l .
To illustrate the notation used in the paper, we will describe it using a simple hierarchy of factors in Figure 3. This example involves two inputs ( x = { i 1 , i 2 } ) and two outputs ( y = { o 1 , o 2 } ). The set of factors f containing all elements from x and y is f = { i 1 , o 1 , i 2 , o 2 } . Overall, there are four factors ( Q 0 = 4 ), two first-level categories ( C 1 = { C 1 ( 1 ) , C 2 ( 1 ) } ; Q 1 = 2 ), and one root corresponding to a second-level category ( C 2 = { C 1 ( 2 ) } ; Q 2 = 1 ). The hierarchy contains two levels of categories ( L = 2 ). The parent category for input i 1 is C 1 ( 1 ) ( p ( i 1 ) = C 1 ( 1 ) ). For C 1 ( 1 ) and C 2 ( 1 ) , the parent category is C 1 ( 2 ) , i.e., p ( C 1 ( 1 ) ) = p ( C 2 ( 1 ) ) = C 1 ( 2 ) . The considered sets of factors A C i ( k ) for example categories are the following: A i 1 = { i 1 } , A C 1 ( 1 ) = { i 1 , o 1 } , A C 2 ( 1 ) = { i 2 , o 2 } , and A C 1 ( 2 ) = f = { i 1 , i 2 , o 1 , o 2 } .
Given a hierarchy of relevant factors and categories, we assign weight w t to each node t except the root. Moreover, we admit specifying the linear constraints for these weights at each hierarchy level. Factors or categories involved in a single constraint must have a common parent. For example, for the considered hierarchy, the constraint can take the form w i 1 2 · w o 1 or w c 1 ( 1 ) w c 2 ( 1 ) , as i 1 and o 1 or C 1 ( 1 ) and C 2 ( 1 ) have the same parent. On the contrary, the example constraint w i 1 w i 2 is not allowed, because i 1 and i 2 have different parents ( p ( i 1 ) p ( i 2 ) ). The space of weight vectors that meet these restrictions is denoted by S w .
To introduce weight restrictions, we consider additional variables ( w ^ q ), representing the aggregated weights of elementary factors in the hierarchy. They are defined as the products of all weights on the path from the analyzed category ( c i ( l ) ) at the hierarchy level l to the analyzed factor f:
w ^ q c i ( l ) = w q · t = 1 , , l 1 t c i ( l ) f A t w t .
For factor i 1 in the considered example, when taking into account the root category ( C 1 ( 2 ) ), the above formula takes the following form w ^ i 1 C 1 ( 2 ) = w i 1 · w C 1 ( 1 ) , and when considering category C 1 ( 1 ) , it is expressed as follows w ^ i 1 C 1 ( 1 ) = w i 1 .
We analyze the efficiency of D M U o in each node of the hierarchy. For category c i ( l ) , such efficiency is defined as follows:
E o c i ( l ) = q A c i ( l ) w ^ q c i ( l ) · u q ( D M U o ) .
The true weights assigned to each hierarchy category c i ( l ) from the set of the indirect children of the analyzed category c j ( k ) are defined as the ratio of the sum of weights of indicators contained in this category and the sum of weights of indicators in the parent category:
w c i ( l ) c j ( k ) = f A c i ( l ) w ^ f c j ( k ) f p A p ( c i ( l ) ) w ^ f p c j ( k ) .
Note that the value of weight w c i ( l ) c j ( k ) is always the same, regardless of the considered category ( c j ( k ) ), so we replace symbol w c i ( l ) c j ( k ) with w c i ( l ) . For example, when considering the root category ( C 1 ( 2 ) ), the weight w i 1 of indicator i 1 in the considered example can be calculated as w i 1 = w ^ i 1 C 1 2 w ^ i 1 C 1 2 + w ^ o 1 C 1 2 , whereas the weight w C 2 ( 1 ) of category C 2 ( 1 ) is w C 2 ( 1 ) = w ^ i 2 C 1 ( 2 ) + w ^ o 2 C 1 ( 2 ) w ^ i 1 C 1 ( 2 ) + w ^ o 1 C 1 ( 2 ) + w ^ i 2 C 1 ( 2 ) + w ^ o 2 C 1 ( 2 ) .

4. Robustness Analysis for Additive Value-Based Efficiency Analysis with a Hierarchical Structure of Factors

The standard value-based efficiency model verifies if each DMU is efficient. Such an analysis builds on the weight vector that is the most advantageous for a given DMU, allowing it to minimize the distance from some efficient DMU. In this section, we introduce a suite of methods that investigate the robustness of efficiency outcomes given all feasible weights. They can be divided into two groups. First, the exact approaches use mathematical programming to find the extreme outcomes for each DMU. In turn, the probabilistic methods estimate the stochastic acceptability indices based on Monte Carlo simulations, reflecting the distributions of possible results. Each group concerns three relevant viewpoints: distances to the efficient unit, ranks, and pairwise preference relations. In what follows, we present the approaches that are flexible enough to determine the relevant results in each hierarchy node.

4.1. Exact Methods

In this section, we present the mathematical programming models that determine the exact robust results. These include extreme (the most and the least advantageous), necessary (observable for all feasible weight vectors), and possible (holding for at least one feasible weight vector) conclusions. Let us first focus on verifying the stability of distances to the efficient unit. The best (minimal) distance d * , o c i ( l ) for D M U o considering category c i ( l ) can be computed by solving the following model:
Minimize d o c i ( l )
s.t.
q A c i ( l ) w ^ q c i ( l ) u q ( D M U k ) q A c i ( l ) w ^ q c i ( l ) u q ( D M U o ) d o c i ( l ) ,   for   k = 1 , , K , d o c i ( l ) 0 , q A c i ( l ) w ^ q c i ( l ) = 1 , w ^ q 0 , q A c i ( l ) , w t = f A t w ^ f c i ( l ) f A p ( t ) w ^ f c i ( l ) S w , f o r t N \ { r o o t } . W
Similarly to the standard efficiency analysis, D M U o with d * , o c i ( l ) = 0 is deemed efficient, given category c i ( l ) , while d * , o c i ( l ) > 0 implies inefficiency.
To compute the worst (maximal) distance d o * , c i ( l ) for D M U o , given category c i ( l ) , we solve the following Mixed-Integer Linear Programming (MILP) model:
Maximize d o c i ( l )
s.t.
q A c i ( l ) w ^ q c i ( l ) u q ( D M U k ) d o c i ( l ) q A c i ( l ) w ^ q c i ( l ) u q ( D M U o ) C ( 1 b k ) ,   for   k = 1 , , K , k = 1 , . . . , K b k = 1 , b k { 0 , 1 } ,   for   k = 1 , 2 , , K , d o c i ( l ) 0 , W ,
where C is a large positive constant. The above model uses binary variables b k , k = 1 , , K , to ensure that d o c i ( l ) is equal to the efficiency difference between D M U o and some D M U k , k = 1 , , K , for which b k = 1 . Maximizing d o c i ( l ) guarantees that we obtain the greatest possible difference observable in the set of feasible weights W . Note that when b k = 0 , the respective constraint is satisfied for all possible variable values; hence, it is relaxed.
The second perspective concerns the bounds of efficiency ranks attained by D M U o . To find the best (minimal) rank R * , o c i ( l ) of D M U o , given category c i ( l ) , we minimize the number of other DMUs with greater efficiencies than E o c i ( l ) :
Minimize 1 + k = 1 , . . . , K ; k o b k
s.t.
q A c i ( l ) w ^ q c i ( l ) u q ( D M U k ) q A c i ( l ) w ^ q c i ( l ) u q ( D M U o ) C b k ,   for   k = 1 , , K ; k o , b k { 0 , 1 } ,   for   k = 1 , 2 , , K ; k o , W .
Note that when b k = 0 , k = 1 , , K , the respective constraint ensures that D M U o is ranked not worse than D M U k since E k c i ( l ) E o c i ( l ) . When b k = 1 , D M U k is ranked better than D M U o , deteriorating its best rank by one.
To obtain the worst (maximal) rank R o * , c i ( l ) for D M U o , given category c i ( l ) , we maximize the number of DMUs with the efficiencies not worse than E o c i ( l ) :
Maximize 1 + k = 1 , . . . , K ; k o b k
s.t.
q A c i ( l ) w ^ q c i ( l ) u q ( D M U o ) q A c i ( l ) w ^ q c i ( l ) u q ( D M U k ) C ( 1 b k ) ,   for   k = 1 , , K ; k o , b k { 0 , 1 } ,   for   k = 1 , 2 , , K ; k o , W .
Note that when b k = 1 , k = 1 , , K , the respective constraint ensures that D M U k is ranked no worse than D M U o since E o c i ( l ) E k c i ( l ) . This deteriorates the worst rank of D M U o by one. When b k = 0 , the respective constraint is satisfied for all variable values; hence, it relaxed.
The third perspective focuses on the pairwise comparisons between DMUs using two relations: necessary and possible. Given the uncertainty of selecting a specific weight vector, the necessary relation can be considered robust. Specifically, D M U o is necessarily preferred to D M U k , given category c i ( l ) ( D M U o E N , c i ( l ) D M U k ), when D M U o is not worse at level c i ( l ) in terms of efficiency than D M U k for all feasible weight vectors. Its truth for pair ( D M U o , D M U k ) and category c i ( l ) can be verified using the following model:
Minimize d o , k c i ( l ) , s . t . q A c i ( l ) w ^ q c i ( l ) u q ( D M U o ) q A c i ( l ) w ^ q c i ( l ) u q ( D M U k ) d o , k c i ( l )   and   W .
Its optimal solution d o , k , * c i ( l ) is equal to the minimal difference between efficiencies of D M U o and D M U k observable in the set of feasible weights W , given category c i ( l ) . If d o , k , * c i ( l ) 0 , then for all feasible weights E o c i ( l ) E k c i ( l ) , and hence D M U o E N , c i ( l ) D M U k . Otherwise, ¬ ( D M U o E N , c i ( l ) D M U k ) because there is at least one feasible weight vector, such that E o c i ( l ) < E k c i ( l ) .
Furthermore, D M U o is possibly preferred to D M U k , given category c i ( l ) ( D M U o E P , c i ( l ) D M U k ), when D M U o is not worse at level c i ( l ) in terms of efficiency than D M U k for at least one feasible weight vector. Its truth for pair ( D M U o , D M U k ) and category c i ( l ) is verified using the following model:
Maximize d o , k c i ( l ) , s . t . q A c i ( l ) w ^ q c i ( l ) u q ( D M U o ) q A c i ( l ) w ^ q u q c i ( l ) ( D M U k ) d o , k c i ( l )   and   W .
Its optimal solution d o , , k c i ( l ) , * is equal to the maximal difference between efficiencies of D M U o and D M U k observable in the set of feasible weights W , given category c i ( l ) . If d o , k c i ( l ) , * 0 , then for at least one feasible weight E o c i ( l ) E k c i ( l ) , and hence, D M U o E P , c i ( l ) D M U k . Otherwise, ¬ ( D M U o E P , c i ( l ) D M U k ) because there is no feasible weight vector, such that E o c i ( l ) E k c i ( l ) .
The relevant properties of the exact robust results given the hierarchical structure are presented in Appendix A. The formulations of example mathematical programming models that support understanding the general formulations are given in Appendix B.

4.2. Simulation-Based Methods

The results determined with mathematical programming are often insufficiently conclusive. In particular, the difference between extreme distances or ranks may be significant, the necessary relation may be poor, and the possible relation may be very rich. If so, it would be helpful to determine the distribution of results observed for the set of feasible weight vectors. Unfortunately, such distribution cannot be computed exactly. However, using Monte Carlo simulations, we can estimate the share of feasible weight space confirming a particular outcome. Specifically, we use the hit-And-run algorithm to generate a predefined number of weight vector samples [27]. We generate weights for all categories and factors while respecting that the sum of weights of categories or factors with the same parent must be equal to one. In the example problem, the sum of the weights of two categories C 1 ( 1 ) and C 2 ( 1 ) must be equal to one ( w C 1 ( 1 ) + w C 2 ( 1 ) = 1 ) , and the sums of weights assigned to the elementary indicators in the same category also need to be one ( w i 1 + w o 1 = 1 and w i 2 + w o 2 = 1 ). Moreover, we obey the provided weight constraints for all hierarchy levels. After generating a predefined number of weight samples, we compute the efficiencies of all DMUs for each. This lets us calculate the relevant stochastic acceptability indices estimating the respective shares of feasible weight vectors.
In what follows, when considering category c i ( l ) and referring to weight vectors, we mean the weights assigned to all categories and factors that are direct or indirect children of c i ( l ) in the hierarchy. The most interesting stochastic acceptabilities are defined as follows:
  • Distance Acceptability Interval Index ( D A I I c i ( l ) ( D M U o , b i ) ) for unit D M U o , interval b i [ 0 , 1 ] , and category c i ( l ) is the share of feasible weight vectors for which E o c i ( l ) belongs to b i . Note that all intervals must be disjoint ( b i b j = , i j ), and their sum must cover the space of possible distances ( b 1 b 2 b z = [ 0 , 1 ] ; z—the number of intervals).
  • Efficiency Rank Acceptability Index ( E R A I c i ( l ) ( D M U o , r ) ) for unit D M U o and rank r is the share of feasible weight vectors for which D M U o attains r-th position in the efficiency ranking of all DMUs given category c i ( l ) .
  • Pairwise Efficiency Outranking Index ( P E O I c i ( l ) ( D M U o , D M U k ) ) for pair ( D M U o , D M U k ) and category c i ( l ) is the share of feasible weight vectors for which D M U o is at least as efficiency as D M U k at level c i ( l ) , i.e., E o c i ( l ) E k c i ( l ) .
Moreover, we compute the expected distance E d to the efficient unit and expected rank E R for each DMU [28]. This is performed by averaging the distances or ranks observed for all samples. Note that by default, we use uniform distribution for weight sampling. However, the weights can be generated from any predefined distribution, but it is hard to define as it requires in-depth knowledge about the specific application domain.
In Appendix C, we illustrate the process of computing the stochastic results on a small sample of weight vectors.

5. Case Study concerning Evaluation of Healthcare System in Poland

This section reports the results of a case study concerning an assessment of the quality of the healthcare system in Poland. This sector faces the challenge of improving the quality of provided services. This can be attained by advancing some indicators reflecting both the system’s functioning and the perception by patients. We consider sixteen voivodeships (provinces) in Poland as DMUs (see Table 1). These administrative areas govern their healthcare independently, so it makes sense to highlight their differences using a uniformly computed set of indicators. Such an evaluation is critical, given the rapid development of new technologies and major transformations in the healthcare sector.
Following [29], we consider three main categories of factors representing desirable characteristics in the complex healthcare system. Two areas—health improvement and financial management—are based on objective indicators and parameters, whereas the system’s evaluation by patients is, to some extent, subjective. In particular, improving health is the ultimate aim of the healthcare system. In this regard, it is relevant to consider the example dimensions of the health status that are affected by how the system is operated, given the ever-growing needs of patients. Financial management is essential in healthcare, as this sector experiences the availability of limited resources. Hence, it is vital to assess the financial situation of medical facilities, the management of infrastructure, and the economic efficiency of treatments and therapies. Finally, consumer satisfaction is becoming more and more important in evaluating the healthcare sector. Thus, it is desirable to consider the quality of services, comfort in using patients’ services, and patient rights.
The hierarchy of inputs and outputs for the case study is presented in Figure 4. Among the nine factors, there are six outputs ( H 1 , H 3 , F 1 , F 3 , S 2 , and S 3 ) and three inputs ( H 2 , F 2 , and S 1 ). The selected indicators are representative of the three dimensions and the viewpoints of the most important stakeholders. The indicators in the health improvement category (H) are representative of the dimensions of preventing diseases ( H 1 ), their exacerbation ( H 2 ), and deaths ( H 3 ). The factors considered in the financial management category stand for the financial situation of healthcare units ( F 1 ) and infrastructure management ( F 2 and F 3 ). The inputs and outputs in the system’s evaluation category (S) represent the waiting time ( S 1 ), official quality system ( S 2 ), and patient satisfaction ( S 3 ). Moreover, we verified that the trends that these indicators confirm also represent other factors that could be considered in the three categories. Note that analysis including over 40 indicators available for assessing the healthcare system in Poland [29] would not make much sense in the context of DEA, as the number of inputs and outputs would be too large compared to the number of DMUs. Typically, such analyses indicate that all or almost all units are efficient, as even the worst performers tend to specialize in some particular aspects. Hence, we opted for an analysis with a reduced—though carefully selected—set of indicators. The performances of the sixteen voivodeships in terms of nine considered factors are given in Table 1.
For all factors, we elicited marginal value functions from experts in the healthcare system in Poland. They are provided in Figure 5. They are decreasing for inputs and increasing for outputs. Moreover, they differ in shape. The function is, e.g., close to linear for H 2 , convex for F 3 , concave for F 2 , and S-shaped for S 2 . Moreover, we incorporated the relative and absolute weight constraints. The category of inhabitants’ health improvement is more important than the two other categories. Hence, we introduced the following constraints: w H w F and w H w S , where w H , w F , and w S are the weights of the three categories. Finally, we wanted to avoid both the minor and dominating roles of any individual factor or category in the analysis. Hence, we restricted the weight of each category to be not less than 0.2 and the weights of second-level elementary factors to the interval [ 0.2 , 0.5 ] . In what follows, we discuss the results attained in the root hierarchy level and for each of the three categories separately.

5.1. Comprehensive Evaluation of the Quality of Healthcare Systems

In this section, we discuss the results of the comprehensive assessment of Polish voivodeships, taking into account all nine indicators. Figure 6 presents the extreme and expected distances to the best unit for each analyzed province. Three voivodeships are efficient: POM, LBU, and WLKP. POM also attains the lowest maximal distance ( 0.105 ), which confirms its most favorable evaluation of the healthcare system for all feasible weights. Moreover, the distances for POM are the most stable, as it is characterized by the narrowest range ( d I , * d * I = 0.105 ). Among the efficient provinces, WLKP has the worst pessimistic distance to the best province. However, its expected distance is better than that of LBU, meaning that for some weight vectors, WLKP performs worse than LBU, but its efficiency score is closer to the best province on average. The worst provinces in the most and the least favorable scenarios are DSL ( d * I = 0.306 , d I , * = 0.522 ) and SL ( d * I = 0.251 , d I , * = 0.526 ). The greatest sensitivity of the distances depending on the selected weight vector is observed for SW, as the width of its distance interval equals 0.355 .
The analysis of extreme distances can be enriched with the distribution of distances over all feasible weight vectors (see Table 2). The efficient provinces (POM, LBU, and WLKP) are the only ones whose distances were not greater than 0.1 for some samples. When considering these three voivodeships, only for LBU, the distance was greater than 0.1 for some marginal share of weights ( 0.8 % ). Hence, these provinces are robustly better than the remaining ones. Among the inefficient units, the most favorable results were attained by ZPM and KP.
Some provinces are characterized by rather stable distance values. For example, for MAZ and WM, for over 95 % of weight vectors, the distance is the interval ( 0.2 , 0.3 ] . On the contrary, the distance for PKR varies more depending on the chosen weight vector. In this case, positive D A I I s were observed for all buckets between 0.1 and 0.5 with the greatest values for the intervals ( 0.3 , 0.4 ] ( D A I I ( P K R , ( 0.3 , 0.4 ] ) = 65.9 % ) and [ 0.2 , 0.3 ] ( D A I I ( P K R , ( 0.2 , 0.3 ] ) = 29.5 % ). The complete ranking determined by the expected distances (see Figure 6) indicates POM ( 0.009 ) and WLKP ( 0.017 ) as the best units and DSL ( 0.408 ) and OPO ( 0.409 ) as the worst.
The results of robustness analysis for efficiency ranks are presented in Figure 7 and Table 3. The three efficient voivodeships attain the first rank in the most favorable scenario. POM and LBU are ranked third in their worst scenario, while WLKP falls fourth in the pessimistic case. These units are also the best, given their expected ranks. In this regard, POM ( 1.543 ) is followed by WLKP ( 1.783 ) and LBU ( 2.674 ). Among the inefficient units, KP is the most advantageous when considering the most favorable ranks ( R * = 2 ). Other inefficient units ranked relatively high in their best scenario are ZPM, WM, LBL, LDZ, SW, MLP, and PKR ( R * = 4 ). However, all inefficient provinces are ranked low in the least favorable scenario. The best maximal rank among them is observed for KP and LBL ( R * = 12 ), while five provinces can be ranked at the bottom (PDL, DSL, OPO, SL, and PKR). Finally, the best expected ranks among inefficient units are attained by ZPM ( 4.962 ), KP ( 5.878 ), and SW ( 6.611 ), while the worst expected positions are associated with DSL ( 15.166 ) and OPO ( 15.195 ).
The analysis of efficiency rank acceptability indices (see Table 3) confirms the superiority of POM over other provinces. It is ranked first for most feasible weight vectors ( 57.7 % ), and it is in the top two for almost 90 % of samples. Similarly, WLKP is at least second for over 84 % of samples, though its most frequent position is second rather than first. In turn, LBU is ranked third for most scenarios ( 72.4 % ). Even though the best possible rank for KP is second, such a position was not observed for any weight vector. The highest for which ERAI for KP is positive is fourth ( 24.8 % ). However, this place is predominantly occupied by ZPM, which is ranked in the interval [ 4 , 7 ] for over 90 % of feasible scenarios.
Table 4 presents the results of exact robustness analysis from the perspective of pairwise comparisons. For clarity of presentation, the necessary relation is also presented in the form of the Hasse diagram in Figure 8. For efficient provinces (POM, WLKP, and LBU), no other unit is necessarily preferred to them. As expected, these provinces are necessarily preferred to the greatest number of other units. POM is robustly better than all thirteen inefficient provinces, and WLKP and LBU prove their superiority over all units but KP. Among the inefficient provinces, KP is necessarily preferred to the highest number (4) of other provinces. Eight voivodeships are not robustly as good as any other unit. The least favorable among them is DSL, which is not even possibly preferred to eight other provinces.
For pairs of voivodeships that are not related by the necessary preference, it is worth analyzing the pairwise efficiency outranking indices (see Table 5). For some, one province proves better for most scenarios (see, e.g., ZPM and PKR with P E O I I ( Z P M , P K R ) = 99.4 % or MAZ and MLP with P E O I I ( M A Z , M L P ) = 99.3 % ). For other pairs, the shares of feasible scenarios confirming the preference in both directions are more balanced (see, e.g., MAZ and LBL with P E O I I ( M A Z , L B L ) = 53.4 % and P E O I I ( L B L , M A Z ) = 46.6 % , or PKR and SL with P E O I I ( P K R , S L ) = 47.6 % and P E O I L ( S L , P K R ) = 52.4 % ).

5.2. The Category of Inhabitants’ Health Improvement

In this section, we focus on the results attained when considering only inputs and outputs from the inhabitants’ health improvement category. In addition, we emphasize the differences with respect to the comprehensive level.
Figure 9 presents the extreme and expected distances to the best unit for all considered voivodeships. WLKP is the only efficient province given this category, so its distance to the best unit always equals zero. Hence, POM and LBU lose the status of efficient units. However, these two voivodeships and ZPM have relatively low distances to the best one: ZPM ( d H [ 0.065 , 0.146 ] ), POM ( d H [ 0.130 , 0.180 ] ), and LBU ( d H [ 0.114 , 0.246 ] ). In general, the widths of distance intervals are notably more precise than when considering all relevant factors jointly. They are also more diverse, encompassing a greater range. In particular, seven provinces have maximal distances greater than 0.6 , with three even exceeding the threshold of 0.7 . At the comprehensive level, this was not observed for any voivodeship.
Among the inefficient provinces, ZPM can be considered the best, as it has the lowest distances to the best unit in both optimistic and pessimistic settings. Moreover, its best expected distance ( 0.109 ) is twice lower than at the comprehensive level, letting it overtake POM and LBU, which were judged efficient in the hierarchy’s root. The other twelve inefficient provinces are significantly worse. The least favorable among them is OPO, with the distance to the best unit in its optimistic scenario equal to 0.73 and an expected distance of 0.754 . In the average case, OPO is directly preceded by MLP ( 0.659 ) and PKR ( 0.652 ). Note that at the comprehensive level, the worst maximal distances were attained by SL, DSL, and PDL. They all prove slightly better regarding inhabitants’ health improvement results.
Table 6 presents the distribution of distances for all voivodeships when considering the inhabitants’ health improvement level. The only two provinces for which this distance is lower than 0.1 are WLKP ( 100 % ) and ZPM ( 33.2 % ). Such a favorable result was not attained by ZPM at the comprehensive level for any feasible weight vector. Then, its distance could drop even above 0.4 . For the two units mentioned above, as well as POM and LBU, all samples confirm distances not higher than 0.2 . Furthermore, for all provinces, we can indicate a single bucket in which the unit’s distance falls for most samples. For example, it is ( 0.2 , 0.3 ] for LBU, ( 0.3 , 0.4 ] for MAZ, and ( 0.7 , 0.8 ] for OPO. For these three provinces, the predominating distance buckets at the comprehensive level were better. However, for other voivodeships, including SW, MLP, and PKR, the most often repeated distance range worsened when limiting the analysis to the inhabitants’ health improvement level.
The extreme and expected efficiency ranks at the inhabitants’ health improvement level are provided in Figure 10. WLKP, as the only efficient province, is always ranked first. Both ZPM and POM are ranked between second and fourth, while for LBU, this range is slightly wider ( [ 2 , 5 ] ). Conversely, only two voivodeships—OPO and PKR—fall to the bottom ranking at any point, and three others (WM, DSL, and MLP) are ranked fifteenth in the least favorable scenario. Finally, for DSL and MAZ, the difference between their extreme ranks is the greatest. For example, DSL is ranked between sixth and fifteenth. According to the expected ranks, WLKP ( E R = 1 ), ZPM ( 2.110 ), POM ( 3.034 ), and LBU ( 3.860 ) are the best, and PKR ( 14.205 ), MLP ( 14.339 ), and OPO ( 15.990 ) are the worst.
When compared to the comprehensive level, the greatest improvement in the attained ranks can be observed for ZPM ( [ 2 , 4 ] rather than [ 4 , 15 ] ) and SL ( [ 4 , 5 ] rather than [ 7 , 16 ] ). On the contrary, the greatest deterioration of possible positions is noted for OPO ( [ 15 , 16 ] rather than [ 8 , 16 ] ) and MLP ( [ 11 , 16 ] rather than [ 4 , 15 ] ). For many provinces, including WM, POL, MAZ, LBL, LDZ, and SW, the ranking intervals got significantly narrower, confirming the lower diversity of results when limiting the scope of the analysis to health improvement.
The distribution of efficiency ranks given the inhabitants’ health improvement level is presented in Table 7. The most stable individual positions were observed for WLKP (1– 100 % ), SL (5– 99.6 % ), OPO (16– 99.0 % ), and ZPM (2– 91.6 % ). Such high acceptabilities were not observed at the comprehensive level for any position and unit. In fact, the greatest share of weights ( 72.4 % ) supported the third position of LBU. Returning to the health improvement category, for some other voivodeships, the vast majority of weights indicate a pair of ranks (e.g., MLP and PKR are ranked 14th or 15th for 92.3 % or 78.3 % samples, respectively). Finally, the ranks of some units are more dependent on the chosen weight vector. For example, KP attained positions between 8th and 12th, with no E R A I exceeding 28 % . Similarly, SW is ranked within the range [ 7 , 12 ] with E R A I s not less than 6.9 % and not greater than 31.1 % . Still, these outcomes exhibit less diversity than at the comprehensive level.
The graph of the necessary relation at the inhabitants’ health improvement level is shown in Figure 11. The robust conclusions are richer than at the comprehensive level. For example, the number of pairs of different provinces that are related by the necessary preference increased from 47 to 85. Moreover, the number of levels in the respective Hasse diagram increased from 3 to 7.
In particular, WLKP is necessarily preferred to all other provinces, and four other voivodeships—POM, ZMP, LBU, and SL—proves to be necessarily better than the remaining eleven units. The worst unit is OPO, which is not necessarily preferred to any other province while being possibly preferred only to PKR. Further, MLP and PKR are necessarily worse than 10 and 9 other provinces, respectively. Finally, SL benefited the most from limiting the scope to the health improvement level, because in the hierarchy’s root, it was not robustly better than any other voivodeship.
The respective P E O I s are given in Table 8. Similarly, as at the comprehensive level, for some pairs, one unit is significantly better than the other (see, e.g., ZPM and POM with P E O I ( Z P M , P O M ) = 94.8 % or LBU and SL with P E O I H ( L B U , S L ) = 98.7 % ). In turn, other pairs are characterized by more balanced stochastic acceptabilities, indicating an advantage of either voivodeship (see, e.g., PKR and MLP with P E O I H ( P K R , M L P ) = 52.2 % or KP and SW with P E O I H ( K P , S W ) = 52.4 % ). However, the absolute values of P O E I s for some pairs differ vastly compared to the hierarchy’s root. For example, POM is necessarily better than ZPM at the comprehensive level, but when considering only health improvement, ZPM attains no worse efficiency for 94.8 % of feasible weights.

5.3. The Category of Financial Management

In this section, we discuss the results attained at the level of financial management. Instead of comparing them to the outcomes at the comprehensive level, we emphasize how managers can use them to improve the relative efficiency of provinces. Similar improvement strategies can be designed for other categories or hierarchy nodes.
Figure 12 presents the extreme and expected distances to the best province when limiting the scope to financial management. The most important result derived from their analysis is the division of provinces into efficient and inefficient. The minimal distance equals zero only for two units: POM and LBU. This means that they are the best performers among the sixteen voivodeships for at least one feasible weight vector. However, LBU has slightly better maximal and expected distances to the best province than POM. When looking at their inputs and outputs within the financial category, they perform equally well ( 51.3 ) on output F 3 . It is the best value among all provinces. Moreover, the other two financial factors are greater for LBU, confirming that it transforms more beds (input F 2 ) into a more significant profit (output F 1 ). These two voivodeships should serve as ultimate peers for the remaining inefficient units in terms of financial management.
Among them, the overall good performers are WLKP ( E d F = 0.1689 ) and MLP ( E d F = 0.2328 ). They do not optimize one specific input or output, but perform decently on all indicators. In the optimistic, pessimistic, and expected scenarios, SL attains the least favorable results with the significantly greatest distances to the best province. In turn, KP has the broadest range of distances ( [ 0.140 , 0.416 ] ), confirming its performance’s sensitivity to the selection of particular priorities. This results from an imbalanced performance profile with a highly favorable value on output F 1 and a relatively poor value on output F 3 . Finally, when complete order is desired, it can be imposed by the expected distances. In this case, LBU ( E d F = 0.010 ) and POM ( E d F = 0.021 ) are safely ranked at the top, and DSL ( E d F = 0.509 ) and SL ( E d F = 0.735 ) are ranked at the bottom.
The distance distribution at the financial management level is presented in Table 9. For both efficient provinces (POM and LBU), the distance is always within the first bucket ( [ 0.0 , 0.1 ] ). Among the inefficient provinces, WLKP confirms its superiority over the remaining units, as for over 99 % samples, its distance from the efficient unit is not greater than 0.2 . The only two other provinces with positive D A I I s for bucket ( 0.1 , 0.2 ] are KP ( 14.2 % ) and MLP ( 4.3 % ). The greatest stability of distances among inefficient provinces is observed for ZPM with D A I I F ( Z P M , ( 0.3 , 0.4 ] ) = 1 . There are only two provinces, PDL and KP, for which most samples confirm no single distance bucket. Furthermore, KP is the only unit with positive D A I I s for more than three buckets ( ( 0.1 , 0.5 ] ).
The results of the robustness analysis for efficiency ranks at the financial management level are presented in Figure 13 (extreme and expected positions) and Table 10 (efficiency rank acceptability indices). They provide additional insights into the comparisons of efficient provinces. Specifically, even if LUB and POM can be ranked first for some weight vectors, the former is ranked first almost twice as often as the latter. This makes it the most favorable province regarding financial management, as additionally confirmed by the expected ranks ( E R F ( L U B ) = 1.377 vs. E R F ( P O M ) = 1.623 ). The remaining provinces are at most third in the best case. Again, WLKP proves to be the most advantageous among them, with E R F ( W L K P ) = 3.107 . Further, KP has the broadest range of efficiency positions, being ranked between third and eleventh. E R A I s confirm that its ranks are rather equally distributed between these extreme positions, with the maximal value for the fifth rank ( E R A I F ( K P , 5 ) = 24.8 % ) and acceptabilities greater than 6 % for all ranks within the range [ 3 , 11 ] . Such great diversity is a consequence of its extreme performances. In fact, it is the best among all provinces on F 1 while being in the bottom three on F 3 . Thus, KP is focused on the profit attained by healthcare institutions rather than on the number of treated patients. This aspect needs to be improved when aiming for higher ranks.
There are three other provinces with relatively wide possible efficiency rank intervals: ZPM ( [ 6 , 12 ] ), PDL ( [ 5 , 11 ] ), and OPO ( [ 9 , 15 ] ). Among them, only ZPM attains a single rank for most samples ( E R A I F ( Z P M , 10 ) = 62.4 % ). Finally, SL is ranked at the bottom regardless of the weight vector. This is related to its greatest value on input F 2 , the lowest value on output F 3 , and a relatively low value on output F 1 . Thus, even if the financial input of SL is the greatest, its outputs are less favorable than for provinces with lesser financial resources. The complete ranking established with the expected efficiency ranks aligns with the one based on expected distances, with LBU, POM, WLKP, and MLP being ranked among the best provinces and SL, DSL, and PKR placed at the bottom.
Figure 14 presents the necessary ranking at the financial management level. POM and LBU are necessarily preferred to all other provinces, confirming their superiority given the financial category. The second level includes KP, WLKP, and MLP, and the lowest one contains PKR, DSL, and SL. They are possibly preferred to 4, 3, and 0 other units, respectively. Moreover, SL is necessarily worse than all other voivodeships.
The pairwise comparisons are useful for analysts particularly familiar with some provinces. For example, the authorities of OPO can compare it to other provinces searching for possible improvements. They can note that OPO is robustly worse than PDL or LBU and robustly better than SL. Notably, the necessary ranking (see Figure 14) is a good starting point to find the improvement paths for provinces. OPO has multiple paths to achieve efficiency. For example, it can take PDL as the first benchmark, follow WLKP, and finally refer to POM or LBU. An alternative improvement path runs through KP and POM.
The analysis of P E O I s (see Table 11) is helpful for pairs related by mutual possible preference. For some, one province attains greater efficiency more often (e.g., P E O I F ( S W , Z P M ) = 91.2 % ). For other pairs, indicating a better voivodeship is more challenging, as the shares of weights confirming the advantage of either unit are similar. An example of such a pair is LDZ and KP with P E O I F ( L D Z , K P ) = 48.3 % and P E O I F ( K P , L D Z ) = 51.7 % .

5.4. The Category of Consumer Satisfaction

In this section, we present the results of provinces’ performance analysis at the level of consumer satisfaction with the healthcare system. To save space, we refer only to the exact and expected results without reporting the complete tables with stochastic acceptabilities.
Figure 15 presents the extreme and expected distances when considering the satisfaction of consumers. There are five efficient provinces with d * S = 0 : POM, KP, LBL, SW, and PKR. This is the greatest number for any node in the considered hierarchy. Among them, two provinces, LBL and PKR, have particularly narrow distance intervals, being close to the efficient units regardless of the weight vector ( d S , * = 0.081 for LBL and d S , * = 0.057 for PKR). They are also the best regarding the expected distance (for PKR— E d S = 0.005 and for LBL— E d S = 0.029 ). In turn, the maximal (worst) possible distances for SW ( 0.285 ) and POM ( 0.307 ) are vastly greater, emphasizing their sensitivity to the selection of a particular weight vector. Among the inefficient units, the best minimal (optimistic) distance is achieved by OPO ( 0.049 ), and the best maximal (pessimistic) distance is attained by WLKP ( 0.312 ). The three provinces that are the worst considering both minimal and maximal distance are PDL ( d * S = 0.454 , d S , * = 0.556 ), DSL ( d * S = 0.449 , d S , * = 0.623 ), and ZPM ( d * S = 0.513 , d S , * = 0.670 ). Their poor performance is also reflected in the bottom ranks according to the expected distances.
The extreme and expected efficiency ranks in consumer satisfaction are presented in Figure 16. Among the five efficient provinces, PKR has the narrowest rank interval; in the least favorable scenario, it is ranked third. On the contrary, even if POM is efficient, it can drop even to the tenth position, making its performance the least stable among all voivodeships. LBU is the best inefficient province when it comes to the best rank ( R * S = 3 ), and the best-ranked inefficient units in the pessimistic settings are WM, LBU, and OPO ( R S , * = 9 ). The worst provinces considering both the minimal and maximal ranks are ZPM ( [ 15 , 16 ] ), PDL ( [ 14 , 15 ] ), and DSL ( [ 14 , 16 ] ). Notably, their rank intervals are very narrow. As far as the expected ranks are concerned, the best province is PKR ( 1.426 ), followed by LBL ( 2.249 ) and KP ( 3.441 ). On the other extreme, PDL ( 14.192 ), DSL ( 14.988 ), and ZPM ( 15.820 ) are the least favorable.
The necessary efficiency preference relation is presented as the Hasse diagram in Figure 17. The provinces form the multiple-level structure, with efficient ones (POM, PKR, LBL, KP, and SW) at the top. Among them, PKR and LBL are necessarily preferred to the most significant number of eleven other provinces, while POM proves robustly better than only six other provinces. Five other voivodeships (WM, LBU, OPO, WLKP, and MLP) are placed in the second level. All of them are necessarily preferred to four or five other provinces. Finally, DSL and ZPM are confirmed as the worst provinces, as they are not necessarily preferred to any other province. In fact, DSL is possibly preferred to only two other units (ZPM and PDL), while ZPM is at least as efficient for at least one weight vector only when compared to DSL.

5.5. Complete Efficiency Rankings of Voivodeships

The expected distances ( E d ) and ranks ( E R ) allow for the construction of a complete ranking of voivodeships. In this section, we compare such orders with the ones obtained with the most commonly used ranking procedures for DEA, i.e., Cross-efficiency (CE) [30] and Super-efficiency (SE) [31]. We adapted them to a value-based additive efficiency model and ran it on each category of indicators.
The rankings generated by all four procedures for the level of inhabitants’ health improvement are provided in Table 12. To quantify the agreement level of such rankings for all hierarchy nodes, we used Kendall’s τ coefficient [28,32]. Its values are shown in Table 13. Note that 1 means the rankings are inverse, whereas 1 denotes a pair of the same rankings.
All four procedures provide highly correlated rankings. The two methods proposed in this paper ( E d and E R ) offer the most similar orders of provinces. The rankings constructed with these two procedures are the same for the comprehensive analysis of the healthcare systems and the customer satisfaction category. For the remaining two categories, Kendall’s τ coefficient equals 0.97 .
The similarity between the rankings based on E d , E R , and CE is between 0.73 for the comprehensive analysis and 0.97 for health improvement and financial management perspectives. The average measure value is 0.904 . When comparing the orders imposed by SE and the two robustness-based methods, Kendall’s τ is between 0.72 and 0.95 ( 0.92 ) for E d ( E R ), with an average value of 0.879 ( 0.863 ). Given that the maximal possible value of Kendall’s τ is 1, the observed similarity levels are very high.

5.6. Discussion

Three provinces, namely POM, WLKP, and LBU, prove to be the best in the comprehensive analysis of the healthcare system in Poland. However, the results for the three subcategories differ. This section discusses the conclusions that can be derived from the cross-category analysis, indicating various provinces’ strong and weak points. For illustrative purposes, we refer to three example voivodeships representing the top (WLKP), middle (PKR), and bottom (OPO) performers in the hierarchy’s root.
When it comes to WLKP, it proves to be the best at the inhabitants’ health improvement level; it is necessarily preferred by POM and LBU for the financial management level, and its ranks are between fifth and twelfth, given the consumer satisfaction perspective. This suggests that despite the decent quality of medical services, there is room for improvement in patient satisfaction and financial management. In particular, managers can conduct some training in soft skills for medical staff to improve consumer assessment of the healthcare system.
PKR is the best province regarding consumer satisfaction, as confirmed by its favorable expected distance and rank. However, when considering the comprehensive results and conclusions drawn for the remaining two categories, it performs poorly. Its expected rank is greater than 13 for inhabitants’ health improvement and financial management categories, while in the hierarchy’s root, its average rank is 12.024 . Hence, the healthcare system managers in this province should focus on improvements in medical decisions and financial management. In turn, other provinces can consider the healthcare system in PKR as the benchmark of proper communication with the consumers.
Such a cross-category analysis can serve as the basis for designing an improvement plan for provinces that proved to be relatively bad in the comprehensive analysis. In particular, the analysis of OPO’s poor performance points to the inhabitants’ health improvement category and financial management. It is the worst province for the former perspective for 99 % of weight vectors, and its expected rank is only 12.549 for the latter category. Hence, managers should first focus on improving inhabitants’ health, due to the high importance of this category in the analysis. Then, they should design a plan for advancing financial management.

6. Conclusions

This paper introduce a novel framework for robustness analysis in the context of additive value-based efficiency analysis with a hierarchical structure. It admits a multiple-layer organization of relevant factors from the most general to the most detailed ones while tolerating both inputs and outputs in the same node. We accept the linear weight restrictions concerning the importance and trade-offs between various factors or subcategories with the common predecessor in the hierarchy. The results can be considered in each hierarchy node, letting the analyst view the comprehensive outcomes and draw conclusions for the subproblems where the relevant factors are limited to concise subsets of inputs and outputs reflecting a particular perspective. The proposed framework can be used in the standard efficiency analysis and the decision contexts requiring the consideration of composite indicators.
We derived the results by considering three perspectives: score-based distances to the efficient unit, ranks, and pairwise preference relations. For each of them, we proposed a pair of methods. The first group was based on mathematical programming, offering the exact, extreme outcomes that can be attained in the set of feasible input/output weights. The other group was based on Monte Carlo simulations, providing the distribution of efficiency outcomes through stochastic acceptabilities that estimate the share of the weight subspaces confirming a given result. These approaches are complementary because the exact outcomes often need more conclusiveness, whereas the stochastic indices—even if approximated with high accuracy—may fail to capture some extreme results.
We illustrated the framework’s applicability by assessing the quality of the healthcare system of sixteen Polish voivodeships. The analysis included nine indicators of different natures and concerned four levels. The comprehensive results were based on all relevant characteristics considered jointly, while the three subproblems captured the perspectives of inhabitants’ health improvement, financial management, and consumer satisfaction. We reported three provinces—Pomorskie (POM), Wielkopolskie (WLKP), and Lubuskie (LBU)—as the most efficient ones. We presented their strong and weak points by referring to the results in all hierarchy nodes. Moreover, we discussed the practical usefulness of the robustness analysis in terms of managerial implications.

Author Contributions

A.L.-K.: conceptualization, methodology, software, investigation, data curation, writing—original draft, visualization; M.K.: conceptualization, methodology, validation, formal analysis, investigation, writing—original draft, supervision, project administration, funding acquisition; W.M.: conceptualization, software, visualization, writing—revision. All authors have read and agreed to the published version of the manuscript.

Funding

Anna Labijak-Kowalska is grateful for the support from the Polish Ministry of Education and Science (grant no. 0311/SBAD/0735). Miłosz Kadziński acknowledges support from the Polish National Science Center under the SONATA BIS project (grant no. DEC-2019/34/E/HS4/00045).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The complete research data produced within the study is contained within this article.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:
AHPAnalytical Hierarchy Process
CECross-efficiency
DEAData Envelopment Analysis
DMUDecision Making Unit
LPLinear Programming
MAVTMulti-Attribute Value Theory
MCDAMultiple Criteria Decision Analysis
MCHPMultiple Criteria Hierarchy Process
MILPMixed-Integer Linear Programming
SESuper-efficiency

Appendix A. Properties of the Exact Robust Results

This section presents the relevant properties of the exact robust results given the hierarchical structure. In particular, when the minimal distances in all children nodes of some more general category are equal to zero, the minimal distance in this category must also be zero.
Proposition A1.
For D M U o D and category c i ( l ) N \ f , if t c h ( c i ( l ) ) : d * , o t = 0 , then d * , o c i ( l ) = 0 .
Similarly, if the minimal rank of D M U o in all children nodes of some more general category is 1 (K), then it needs to be ranked first (last) in this category in the best case.
Proposition A2.
For D M U o and category c i ( l ) N \ f , if t c h ( c i ( l ) ) : R * , o t = 1 then R * , o c i ( l ) = 1 .
Proposition A3.
For D M U o and category c i ( l ) i n N \ f , if t c h ( c i ( l ) ) : R * , o t = K , then R * , o c i ( l ) = K .
Moreover, if the maximal rank of D M U o in all children nodes of some more general category is 1, then it needs to be ranked first in this category, even in the worst case.
Proposition A4.
For D M U o and category c i ( l ) N \ f , if t c h ( c i ( l ) ) : R o * , t = 1 , then R o * , c i ( l ) = 1 .
When D M U o is necessarily preferred to D M U k in all children nodes of some more general category, then D M U o must be necessarily preferred to D M U k , given this category.
Proposition A5.
For pair ( D M U o , D M U k ) and category c i ( l ) N \ f , if t c h ( c i ( l ) ) : D M U o E N , t D M U k , then D M U o E N , c i ( l ) D M U k .
Note that E N , c i ( l ) is a partial preorder (i.e., transitive and reflexive). When D M U o is not possibly preferred to D M U k in all children nodes of some more general category, then D M U o is not possibly preferred to D M U k , given this category.
Proposition A6.
For pair ( D M U o , D M U k ) and category c i ( l ) N \ f , if t c h ( c i ( l ) ) : ¬ ( D M U o E P , t D M U k ) , then ¬ ( D M U o E P , c i ( l ) D M U k ) .
Moreover, when D M U o is necessarily preferred to D M U k in all children nodes of some more general category except one node for which it is possibly preferred, then D M U o needs to be possibly preferred to D M U k , given this category.
Proposition A7.
For pair ( D M U o , D M U k ) and category c i ( l ) N \ f , if t c h ( c i ( l ) ) \ a : D M U o E N , t D M U k D M U o E P , a D M U k , then D M U o E P , c i ( l ) D M U k .
Note that E P , c i ( l ) is negatively transitive and strongly complete. Moreover, the truth of the necessary preference implies the truth of the possible preference ( E N , c i ( l ) E P , c i ( l ) ).

Appendix B. Formulations of Example Mathematical Programming Models for Computing the Exact Robust Results

This section illustrates mathematical programming models for computing the exact robust results. They support understanding the general formulations presented in Section 4. We consider a hierarchical structure involving four indicators as presented in Figure 3. The input and output values of four DMUs and the indicator bounds are shown in Table A1. For simplicity, we assume that the marginal value functions are linear to make them easily computable. For example, for D M U 1 and input i 1 , the marginal value is calculated as u i 1 ( D M U 1 ) = u i 1 ( 9 ) = 10 9 10 0 = 0.1 . Analogously, for D M U 3 and output o 2 , the value of marginal function is equal to u o 2 ( D M U 3 ) = u o 2 ( 6 ) = 6 5 10 5 = 0.2 . Moreover, the following weight constraints are considered: w C 1 ( 1 ) w C 1 ( 1 ) , w C 2 ( 1 ) 0.1 , w i 1 0.2 , and w o 2 0.6 .
Table A1. Input/output values for DMUs and indicators’ bounds in the illustrative problem.
Table A1. Input/output values for DMUs and indicators’ bounds in the illustrative problem.
DMU i 1 o 1 i 2 o 2
D M U 1 9587
D M U 2 2855
D M U 3 5676
D M U 4 1298
min0055
max10101010
All formulations concern category C 1 ( 2 ) (i.e., the hierarchy’s root). Let us first provide the model for computing the minimal distance of D M U 1 to the best unit:
Minimize d 1
s.t.
w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) ( w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) ) d 1 w ^ i 1 C 1 ( 2 ) u i 1 ( 2 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 8 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 5 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 5 ) ( w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) ) d 1 w ^ i 1 C 1 ( 2 ) u i 1 ( 5 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 6 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 7 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 6 ) ( w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) ) d 1 w ^ i 1 C 1 ( 2 ) u i 1 ( 1 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 2 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 9 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 8 ) ( w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) ) d 1 d 1 0 w ^ i 1 C 1 ( 2 ) + w ^ o 1 C 1 ( 2 ) + w ^ i 2 C 1 ( 2 ) + w ^ o 2 C 1 ( 2 ) = 1 w ^ i 1 C 1 ( 2 ) + w ^ o 1 C 1 ( 2 ) w ^ i 1 C 1 ( 2 ) + w ^ o 1 C 1 ( 2 ) + w ^ i 2 C 1 ( 2 ) + w ^ o 2 C 1 ( 2 ) w ^ i 2 C 1 ( 2 ) + w ^ o 2 C 1 ( 2 ) w ^ i 1 C 1 ( 2 ) + w ^ o 1 C 1 ( 2 ) + w ^ i 2 C 1 ( 2 ) + w ^ o 2 C 1 ( 2 ) w ^ i 2 C 1 ( 2 ) + w ^ o 2 C 1 ( 2 ) w ^ i 1 C 1 ( 2 ) + w ^ o 1 C 1 ( 2 ) + w ^ i 2 C 1 ( 2 ) + w ^ o 2 C 1 ( 2 ) 0.1 w ^ i 1 C 1 ( 2 ) w ^ i 1 C 1 ( 2 ) + w ^ o 1 C 1 ( 2 ) 0.2 w ^ o 2 C 1 ( 2 ) w ^ i 2 C 1 ( 2 ) + w ^ o 2 C 1 ( 2 ) 0.6 w ^ i 1 C 1 ( 2 ) 0 , w ^ o 1 C 1 ( 2 ) 0 , w ^ i 2 C 1 ( 2 ) 0 , w ^ o 2 C 1 ( 2 ) 0 . W C 1 ( 2 )
In the above model, the first four constraints ensure that the optimized distance will correspond to the greatest distance of D M U 1 to other DMU. The fifth constraint ensures that the distance is non-negative. The constraint set W C 1 ( 2 ) corresponds to the weight restrictions. In particular, the first constraint ensures that the sum of weights equals one, whereas the last guarantees that all weights are non-negative. In turn, to find the maximal distance of D M U 1 , we need to solve the following model:
Maximize d 1
s.t.
w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) d 1 w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) C ( 1 b 1 ) w ^ i 1 C 1 ( 2 ) u i 1 ( 2 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 8 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 5 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 5 ) d 1 w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) C ( 1 b 2 ) w ^ i 1 C 1 ( 2 ) u i 1 ( 5 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 6 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 7 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 6 ) d 1 w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) C ( 1 b 3 ) w ^ i 1 C 1 ( 2 ) u i 1 ( 1 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 2 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 9 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 8 ) d 1 w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) C ( 1 b 4 ) b 1 + b 2 + b 3 + b 4 = 1 b 1 , b 2 , b 3 , b 4 { 0 , 1 } d 1 0 , W C 1 ( 2 )
The following model allows us to find the minimal (best) efficiency rank for D M U 1 :
Minimize 1 + b 2 + b 3 + b 4
s.t.
w ^ i 1 C 1 ( 2 ) u i 1 ( 2 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 8 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 5 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 5 ) ( w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) ) C b 2 w ^ i 1 C 1 ( 2 ) u i 1 ( 5 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 6 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 7 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 6 ) ( w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) ) C b 3 w ^ i 1 C 1 ( 2 ) u i 1 ( 1 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 2 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 9 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 8 ) ( w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) ) C b 4 b 2 , b 3 , b 4 { 0 , 1 } W C 1 ( 2 )
The maximal (worst) efficiency rank for D M U 1 can be determined by solving the following model:
Maximize 1 + b 2 + b 3 + b 4
s.t.
w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) ( w ^ i 1 C 1 ( 2 ) u i 1 ( 2 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 8 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 5 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 5 ) ) C ( 1 b 2 ) w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) ( w ^ i 1 C 1 ( 2 ) u i 1 ( 5 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 6 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 7 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 6 ) ) C ( 1 b 3 ) w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) ( w ^ i 1 C 1 ( 2 ) u i 1 ( 1 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 2 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 9 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 8 ) ) C ( 1 b 4 ) b 2 , b 3 , b 4 { 0 , 1 } W C 1 ( 2 )
When referring to pairwise preference relations, we consider an ordered pair ( D M U 1 , D M U 2 ) . The truth of the necessary preference relation can be verified by solving the following model:
Minimize d 1 , 2
s.t.
w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) ( w ^ i 1 C 1 ( 2 ) u i 1 ( 2 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 8 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 5 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 5 ) ) d 1 , 2 W C 1 ( 2 )
If the minimal distance d 1 , 2 is not lesser than zero, then D M U 1 E N , C 1 ( 2 ) D M U 2 . The following model allows us to verify the truth of the possible preference relation:
Maximize d 1 , 2
s.t.
w ^ i 1 C 1 ( 2 ) u i 1 ( 9 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 5 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 8 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 7 ) ( w ^ i 1 C 1 ( 2 ) u i 1 ( 2 ) + w ^ o 1 C 1 ( 2 ) u o 1 ( 8 ) + w ^ i 2 C 1 ( 2 ) u i 2 ( 5 ) + w ^ o 2 C 1 ( 2 ) u o 2 ( 5 ) ) d 1 , 2 W C 1 ( 2 ) .
If the maximal value of d 1 , 2 is greater or equal to zero, then D M U 1 E P , C 1 ( 2 ) D M U 2 .

Appendix C. Computation of Stochastic Acceptability Indices

For illustrative purposes, we present the steps of computing the stochastic acceptability indices for the exemplary DMUs from Appendix B considering the root category ( C 1 ( 2 ) ). Firstly, we use the hit-and-run algorithm to generate the weight samples for all children of C 1 ( 2 ) . Table A2 presents the five example samples. Let us emphasize that to obtain reliable estimates of stochastic acceptabilities, in practice, one uses a few thousand such samples. Secondly, we compute a value-based efficiency score for each considered DMU (see Table A3). Based on the efficiency scores, we determine the distance of each D M U o to the best one as the difference between the maximal efficiency score obtained by any D M U k for a given sample and the efficiency score of D M U o . For example, for sample 1 and D M U 3 , such distance equals 0.773 0.542 = 0.231 . The efficiency rank of D M U o is equal to the number of other DMUs, for which the efficiency score is better than that of D M U o (for a given sample), increased by one. For example, for sample 1, D M U 2 and D M U 3 attained an efficiency score greater than D M U 4 , ranking it 3rd. The distances to the best DMU and efficiency ranks for all exemplary DMUs and samples are presented in Table A3.
Table A2. Five example weight vectors obtained with Monte Carlo simulation.
Table A2. Five example weight vectors obtained with Monte Carlo simulation.
Sample12345
w i 1 0.330.520.560.810.7
w o 1 0.670.480.440.190.3
w i 2 0.710.530.460.680.46
w o 2 0.290.470.540.320.54
w C 1 ( 1 ) 0.70.680.750.530.69
w C 2 ( 1 ) 0.30.320.250.470.31
Having determined the distances to the best DMU and efficiency ranks, we compute the stochastic indices. D A I I is the share of the weight vectors for which the distance to the best DMU is within a given bucket. For illustrative purposes, we use only four buckets. For example, there are four out of five samples for which the distance of D M U 4 to the best unit is in the interval b 1 = [ 0 , 0.25 ] , so D A I I ( D M U 4 , [ 0 , 0.25 ] ) = 0.8 . Similarly, we calculate E R A I s as the share of samples for which a given DMU is ranked r-th. For example, D M U 3 is ranked 2nd only for sample 1, so E R A I ( D M U 3 , 2 ) = 0.2 . Finally, P E O I for an ordered pair of units is computed as the share of samples for which the first DMU is at least as efficient as the other. For example, D M U 4 is not worse than D M U 3 for all samples except the first one, so P E O I ( D M U 4 , D M U 3 ) = 0.8 . The results obtained by individual samples are averaged, providing the estimates of the expected distances E d to the best DMU and expected efficiency ranks E R (see Table A3).
Table A3. Efficiency scores E, distances d, and ranks R for the considered DMUs obtained with five sampled weight vectors.
Table A3. Efficiency scores E, distances d, and ranks R for the considered DMUs obtained with five sampled weight vectors.
SampleResult DMU 1 DMU 2 DMU 3 DMU 4
1E0.3780.7730.5420.397
d0.39500.2310.377
R4123
2E0.3270.7140.5040.508
d0.38700.2090.206
R4132
3E0.3070.7150.5040.548
d0.40800.2110.167
R4132
4E0.2810.7440.4970.561
d0.46200.2470.183
R4132
5E0.2760.6950.4850.605
d0.41900.2100.090
R4132
E d 0.2510.2020.3040.377
E R 4.01.02.82.2

References

  1. Charnes, A.; Cooper, W.W.; Rhodes, E. Measuring the efficiency of decision making units. Eur. J. Oper. Res. 1978, 2, 429–444. [Google Scholar] [CrossRef]
  2. Liu, J.S.; Lu, L.Y.; Lu, W.M.; Lin, B.J. A survey of DEA applications. Omega 2013, 41, 893–902. [Google Scholar] [CrossRef]
  3. Krejnus, M.; Stofkova, J.; Stofkova, K.R.; Binasova, V. The Use of the DEA Method for Measuring the Efficiency of Electronic Public Administration as Part of the Digitization of the Economy and Society. Appl. Sci. 2023, 13, 3672. [Google Scholar] [CrossRef]
  4. Wang, L.; Lu, F.; Han, B.; Zhang, Q.; Zhang, C. Simulation-Based Optimization of Transport Efficiency of an Urban Rail Transit Network. Appl. Sci. 2023, 13, 1471. [Google Scholar] [CrossRef]
  5. Emrouznejad, A.; Parker, B.R.; Tavares, G. Evaluation of research in efficiency and productivity: A survey and analysis of the first 30 years of scholarly literature in DEA. Socio-Econ. Plan. Sci. 2008, 42, 151–157. [Google Scholar] [CrossRef]
  6. Cooper, W.W.; Seiford, L.M.; Zhu, J. Data envelopment analysis: History, models, and interpretations. In Handbook on Data Envelopment Analysis; Springer: Berlin/Heidelberg, Germany, 2011; pp. 1–39. [Google Scholar]
  7. Charnes, A.; Cooper, W.W.; Golany, B.; Seiford, L.; Stutz, J. Foundations of data envelopment analysis for Pareto-Koopmans efficient empirical production functions. J. Econom. 1985, 30, 91–107. [Google Scholar] [CrossRef]
  8. Gouveia, M.C.; Dias, L.C.; Antunes, C.H. Additive DEA based on MCDA with imprecise information. J. Oper. Res. Soc. 2008, 59, 54–63. [Google Scholar] [CrossRef]
  9. de Almeida, P.; Dias, L. Value-based DEA models: Application-driven developments. J. Oper. Res. Soc. 2012, 63, 16–27. [Google Scholar] [CrossRef]
  10. Keeney, R.L.; Raiffa, H.; Meyer, R.F. Decisions with Multiple Objectives: Preferences and Value Trade-Offs; Cambridge University Press: Cambridge, UK, 1993. [Google Scholar]
  11. Alidrisi, H. DEA-Based PROMETHEE II Distribution-Center Productivity Model: Evaluation and Location Strategies Formulation. Appl. Sci. 2021, 11, 9567. [Google Scholar] [CrossRef]
  12. Bagherikahvarin, M.; De Smet, Y. A ranking method based on DEA and PROMETHEE II (a rank based on DEA & PR.II). Measurement 2016, 89, 333–342. [Google Scholar]
  13. Saaty, T.L. What is the Analytic Hierarchy Process? In Mathematical Models for Decision Support; Springer: Berlin/Heidelberg, Germany, 1988; pp. 109–121. [Google Scholar]
  14. Corrente, S.; Greco, S.; Słowiński, R. Multiple Criteria Hierarchy Process in Robust Ordinal Regression. Decis. Support Syst. 2012, 53, 660–674. [Google Scholar] [CrossRef]
  15. Corrente, S.; Greco, S.; Słowiński, R. Multiple Criteria Hierarchy Process with ELECTRE and PROMETHEE. Omega 2013, 41, 820–846. [Google Scholar] [CrossRef]
  16. Del Vasto-Terrientes, L.; Valls, A.; Slowinski, R.; Zielniewicz, P. ELECTRE-III-H: An outranking-based decision aiding method for hierarchically structured criteria. Expert Syst. Appl. 2015, 42, 4910–4926. [Google Scholar] [CrossRef]
  17. Meng, W.; Zhang, D.; Qi, L.; Liu, W. Two-level DEA approaches in research evaluation. Omega 2008, 36, 950–957. [Google Scholar] [CrossRef]
  18. Kao, C. A linear formulation of the two-level DEA model. Omega 2008, 36, 958–962. [Google Scholar] [CrossRef]
  19. Shen, Y.; Hermans, E.; Ruan, D.; Wets, G.; Brijs, T.; Vanhoof, K. A generalized multiple layer data envelopment analysis model for hierarchical structure assessment: A case study in road safety performance evaluation. Expert Syst. Appl. 2011, 38, 15262–15272. [Google Scholar] [CrossRef]
  20. Pakkar, M.S. An integrated approach based on DEA and AHP. Comput. Manag. Sci. 2015, 12, 153–169. [Google Scholar] [CrossRef]
  21. Pakkar, M.S. A hierarchical aggregation approach for indicators based on data envelopment analysis and analytic hierarchy process. Systems 2016, 4, 6. [Google Scholar] [CrossRef]
  22. Amini, M.R.; Azar, A.; Eskandari, H.; Wanke, P.F. A generalized fuzzy Multiple-Layer NDEA: An application to performance-based budgeting. Appl. Soft Comput. 2021, 100, 106984. [Google Scholar] [CrossRef]
  23. Kadziński, M.; Labijak, A.; Napieraj, M. Integrated framework for robustness analysis using ratio-based efficiency model with application to evaluation of Polish airports. Omega 2017, 67, 1–18. [Google Scholar] [CrossRef]
  24. Labijak-Kowalska, A.; Kadziński, M.; Spychała, I.; Dias, L.C.; Fiallos, J.; Patrick, J.; Michalowski, W.; Farion, K. Performance evaluation of emergency department physicians using robust value-based additive efficiency model. Int. Trans. Oper. Res. 2023, 30, 503–544. [Google Scholar] [CrossRef]
  25. Miszczyńska, K.; Miszczyński, P.M. Measuring the efficiency of the healthcare sector in Poland—A window-DEA evaluation. Int. J. Product. Perform. Manag. 2022, 71, 2743–2770. [Google Scholar] [CrossRef]
  26. Zakowska, I.; Godycki-Cwirko, M. Data envelopment analysis applications in primary health care: A systematic review. Fam. Pract. 2019, 37, 147–153. [Google Scholar] [CrossRef] [PubMed]
  27. Ciomek, K.; Kadziński, M. Polyrun: A Java library for sampling from the bounded convex polytopes. SoftwareX 2021, 13, 100659. [Google Scholar] [CrossRef]
  28. Labijak-Kowalska, A.; Kadziński, M. Experimental comparison of results provided by ranking methods in Data Envelopment Analysis. Expert Syst. Appl. 2021, 173, 114739. [Google Scholar] [CrossRef]
  29. Kozierkiewicz, A.; Natkaniec, M.; Megas, B.; Gilewski, D.; Ignatowicz, M.; Waśko, B. Indeks Sprawności Ochrony Zdrowia 2018; Technical Report; PricewaterhouseCoopers: Warsaw, Poland, 2019. (In Polish) [Google Scholar]
  30. Sexton, T.R.; Silkman, R.H.; Hogan, A.J. Data envelopment analysis: Critique and extensions. New Dir. Program Eval. 1986, 1986, 73–105. [Google Scholar] [CrossRef]
  31. Andersen, P.; Petersen, N.C. A procedure for ranking efficient units in data envelopment analysis. Manag. Sci. 1993, 39, 1261–1264. [Google Scholar] [CrossRef]
  32. Kendall, M.G. A new measure of rank correlation. Biometrika 1938, 30, 81–93. [Google Scholar] [CrossRef]
Figure 1. Paper’s main contributions.
Figure 1. Paper’s main contributions.
Applsci 13 06406 g001
Figure 2. A hierarchical structure of inputs and outputs.
Figure 2. A hierarchical structure of inputs and outputs.
Applsci 13 06406 g002
Figure 3. An example hierarchical structure of inputs and outputs.
Figure 3. An example hierarchical structure of inputs and outputs.
Applsci 13 06406 g003
Figure 4. A hierarchy of indicators considered in the quality evaluation of healthcare systems.
Figure 4. A hierarchy of indicators considered in the quality evaluation of healthcare systems.
Applsci 13 06406 g004
Figure 5. The marginal value functions associated with the inputs and outputs considered in the case study.
Figure 5. The marginal value functions associated with the inputs and outputs considered in the case study.
Applsci 13 06406 g005
Figure 6. Extreme and expected distances to the best unit for Polish voivodeships in the comprehensive analysis of the healthcare system.
Figure 6. Extreme and expected distances to the best unit for Polish voivodeships in the comprehensive analysis of the healthcare system.
Applsci 13 06406 g006
Figure 7. Extreme and expected ranks for Polish voivodeships in the comprehensive analysis of the healthcare system.
Figure 7. Extreme and expected ranks for Polish voivodeships in the comprehensive analysis of the healthcare system.
Applsci 13 06406 g007
Figure 8. The Hasse diagram of the necessary efficiency preference relation in the comprehensive analysis of the healthcare system.
Figure 8. The Hasse diagram of the necessary efficiency preference relation in the comprehensive analysis of the healthcare system.
Applsci 13 06406 g008
Figure 9. Extreme and expected distances to the best unit for Polish voivodeships when considering inhabitants’ health improvement level.
Figure 9. Extreme and expected distances to the best unit for Polish voivodeships when considering inhabitants’ health improvement level.
Applsci 13 06406 g009
Figure 10. Extreme and expected ranks for Polish voivodeships when considering inhabitants’ health improvement level.
Figure 10. Extreme and expected ranks for Polish voivodeships when considering inhabitants’ health improvement level.
Applsci 13 06406 g010
Figure 11. The Hasse diagram of the necessary efficiency preference relation when considering inhabitants’ health improvement level.
Figure 11. The Hasse diagram of the necessary efficiency preference relation when considering inhabitants’ health improvement level.
Applsci 13 06406 g011
Figure 12. Extreme and expected distances to the best unit for Polish voivodeships when considering financial management level.
Figure 12. Extreme and expected distances to the best unit for Polish voivodeships when considering financial management level.
Applsci 13 06406 g012
Figure 13. Extreme and expected ranks for Polish voivodeships when considering financial management level.
Figure 13. Extreme and expected ranks for Polish voivodeships when considering financial management level.
Applsci 13 06406 g013
Figure 14. The Hasse diagram of the necessary efficiency preference relation when considering financial management level.
Figure 14. The Hasse diagram of the necessary efficiency preference relation when considering financial management level.
Applsci 13 06406 g014
Figure 15. Extreme and expected distances to the best unit for Polish voivodeships when considering consumer satisfaction level.
Figure 15. Extreme and expected distances to the best unit for Polish voivodeships when considering consumer satisfaction level.
Applsci 13 06406 g015
Figure 16. Extreme and expected ranks for Polish voivodeships when considering consumer satisfaction level.
Figure 16. Extreme and expected ranks for Polish voivodeships when considering consumer satisfaction level.
Applsci 13 06406 g016
Figure 17. The Hasse diagram of the necessary efficiency preference relation when considering consumer satisfaction level.
Figure 17. The Hasse diagram of the necessary efficiency preference relation when considering consumer satisfaction level.
Applsci 13 06406 g017
Table 1. Values of inputs and outputs related to the healthcare systems in sixteen Polish voivodeships [29] (data publicly available at https://www.pwc.pl/pl/publikacje/2019/indeks-sprawnosci-ochrony-zdrowia-2018.html, accessed on 22 May 2023).
Table 1. Values of inputs and outputs related to the healthcare systems in sixteen Polish voivodeships [29] (data publicly available at https://www.pwc.pl/pl/publikacje/2019/indeks-sprawnosci-ochrony-zdrowia-2018.html, accessed on 22 May 2023).
VoivodeshipShort Name H 1 H 2 H 3 F 1 F 2 F 3 S 1 S 2 S 3
ZachodniopomorskieZPM44.444.0517.6−1.546.146.919.233.559.30
PomorskiePOM45.315.3621.30.0838.951.327.043.8233.33
Warmińsko−MazurskieWM43.347.1115.7−1.0944.847.822.193.7222.73
PodlaskiePDL37.546.9315.82.349.443.419.473.6111.11
LubuskieLBU50.215.8318.01.942.051.318.323.8413.04
WielkopolskieWLKP47.884.3625.7−0.343.450.515.163.6917.24
Kujawsko−PomorskieKP39.906.8015.73.846.541.822.363.7733.33
MazowieckieMAZ38.595.3216.7−1.5747.447.820.263.6220.56
LubelskieLBL45.176.748.10.4151.843.013.403.7720.37
DolnośląskieDSL45.047.6613.8−1.7750.843.824.553.5717.28
OpolskieOPO33.716.8810.0−0.3644.140.923.033.6735.71
ŁódzkieLDZ42.005.886.01.0250.647.319.903.7815.15
ŚląskieSL40.704.5413.9−1.6355.539.122.023.7516.88
ŚwiętokrzyskieSW41.145.797.80.1749.547.212.443.5729.17
MałopolskieMLP32.225.619.10.7943.744.318.273.6425.84
PodkarpackiePKR38.735.761.9−4.3148.045.814.393.8420.51
Table 2. Distribution of distances to the best unit for Polish voivodeships in the comprehensive analysis of the healthcare system.
Table 2. Distribution of distances to the best unit for Polish voivodeships in the comprehensive analysis of the healthcare system.
DAII[0.0, 0.1](0.1, 0.2](0.2, 0.3](0.3, 0.4](0.4, 0.5](0.5, 0.6](0.6, 0.7](0.7, 0.8](0.8, 0.9](0.9, 1.0]
ZPM00.5230.4750.002000000
POM1000000000
WM00.0060.9560.038000000
PDL000.0060.7140.28000000
LBU0.9920.00800000000
WLKP1000000000
KP00.1600.7980.042000000
MAZ00.0010.9630.036000000
LBL00.0740.7290.197000000
DSL0000.4060.59400000
OPO0000.4070.5870.0060000
LDZ000.5990.401000000
SL000.2540.7320.01400000
SW00.1270.7620.111000000
MLP000.3070.6750.01800000
PKR00.0020.2950.6590.04400000
Table 3. Distribution of ranks for Polish voivodeships in the comprehensive analysis of the healthcare system.
Table 3. Distribution of ranks for Polish voivodeships in the comprehensive analysis of the healthcare system.
ERAI12345678910111213141516
ZPM0000.5940.1440.0950.0720.0650.0280.002000000
POM0.5770.3030.1200000000000000
WM0000.0290.1920.2100.1910.1830.1030.0420.0300.0200000
PDL0000000000.0020.0090.0720.1140.4430.2880.072
LBU0.0500.2260.7240000000000000
WLKP0.3730.4710.1560000000000000
KP0000.2480.2450.1740.1620.0990.0390.0230.01000000
MAZ00000.1010.1610.2150.1710.2200.0960.0330.0030000
LBL0000.0330.0840.1510.1490.1580.2560.1040.0440.0170.004000
DSL00000000000.0030.0220.0420.1400.3250.468
OPO00000000000.0010.0010.0170.2170.3110.453
LDZ000000.0070.0280.0690.1390.3600.2780.0900.0270.00200
SL00000.0070.0100.0280.0320.0410.1200.1500.1840.3350.0680.0220.003
SW0000.0960.2270.1900.1460.1950.0980.0400.00800000
MLP0000000.0030.0180.0520.1170.2480.2660.2540.0330.0090
PKR000000.0020.0060.0100.0240.0940.1860.3250.2070.0970.0450.004
Table 4. The necessary (N) and possible (P) efficiency preference relations for pairs of Polish voivodeships in the comprehensive analysis of the healthcare system.
Table 4. The necessary (N) and possible (P) efficiency preference relations for pairs of Polish voivodeships in the comprehensive analysis of the healthcare system.
VoivodeshipZPMPOMWMPDLLBUWLKPKPMAZLBLDSLOPOLDZSLSWMLPPKR
ZPMN PP PPPNPPPPPP
POMNNNNPPNNNNNNNNNN
WMP NP PPPNNPPPPP
PDLP PN PPPPPPPPP
LBUNPNNNPPNNNNNNNNN
WLKPNPNNPNPNNNNNNNNN
KPP PNPPNPPNNPNPPP
MAZP PP PNPPPPPPPP
LBLP PP PPNNPPNPPP
DSL P P NPPP PP
OPOP P PPPNPPPPP
LDZP PP PPPPPNPPPP
SLP PP P PPPNPPP
SWP PP PPPNPPPNPP
MLPP PP PPPPPPPPNP
PKRP PP PPPPPPPPPN
Table 5. Pairwise efficiency outranking indices for pairs of Polish voivodeships in the comprehensive analysis of the healthcare system.
Table 5. Pairwise efficiency outranking indices for pairs of Polish voivodeships in the comprehensive analysis of the healthcare system.
PEOIZPMPOMWMPDLLBUWLKPKPMAZLBLDSLOPOLDZSLSWMLPPKR
ZPM100.8351000.6900.9620.852110.98910.79110.994
POM11110.8700.6251111111111
WM0.165011000.2860.6090.630110.8900.9180.4810.9600.967
PDL0001000000.7600.7480.0060.09500.0390.146
LBU10.1301110.1811111111111
WLKP10.375110.81911111111111
KP0.31000.71410010.7580.805110.9530.9740.6510.9960.990
MAZ0.03800.3911000.24210.534110.8500.9810.3780.9930.958
LBL0.14800.3701000.1950.4661110.8790.9090.2950.8980.988
DSL0000.2400000010.4870.0010.01600.0520.061
OPO0000.252000000.51310.0020.023000.020
LDZ0.01100.1100.994000.0470.1500.1210.9990.99810.7450.0510.7570.849
SL000.0820.905000.0260.0190.0910.9840.9770.25510.0490.4890.524
SW0.20900.5191000.3490.6220.705110.9490.95110.9970.992
MLP000.0400.961000.0040.0070.1020.94810.2430.5110.00310.568
PKR0.00600.0330.854000.0100.0420.0120.9390.9800.1510.4760.0080.4321
Table 6. Distribution of distances to the best unit for Polish voivodeships when considering inhabitants’ health improvement level.
Table 6. Distribution of distances to the best unit for Polish voivodeships when considering inhabitants’ health improvement level.
DAII[0.0, 0.1](0.1, 0.2](0.2, 0.3](0.3, 0.4](0.4, 0.5](0.5, 0.6](0.6, 0.7](0.7, 0.8](0.8, 0.9](0.9, 1.0]
ZPM0.3320.66800000000
POM0100000000
WM0000.0550.7270.2180000
PDL000000.6580.342000
LBU00.7130.2870000000
WLKP1000000000
KP00000.3410.6590000
MAZ0000.8170.18300000
LBL00000.3420.6180.040000
DSL0000.0330.5100.4330.024000
OPO0000000100
LDZ00000.1500.7100.140000
SL000.4330.567000000
SW00000.2560.7440000
MLP000000.0880.7150.19700
PKR000000.1250.7080.16700
Table 7. Distribution of ranks for Polish voivodeships when considering inhabitants’ health improvement level.
Table 7. Distribution of ranks for Polish voivodeships when considering inhabitants’ health improvement level.
ERAI12345678910111213141516
ZPM00.9160.0580.026000000000000
POM00.0540.8580.088000000000000
WM000000.0290.7050.1100.0930.063000000
PDL000000000.0030.1790.0960.0750.4030.1420.1020
LBU00.0300.0840.8820.00400000000000
WLKP1000000000000000
KP00000000.2120.2720.1820.0870.2470000
MAZ000000.9320.0130.0460.0090000000
LBL0000000.0490.0360.2940.2060.1770.1760.0490.01300
DSL000000.0390.0390.3990.2010.0700.1420.0640.0190.0260.0010
OPO000000000000000.0100.990
LDZ00000000.1200.0590.0850.1590.2810.296000
SL0000.0040.99600000000000
SW0000000.1940.0770.0690.2150.3110.1340000
MLP00000000000.0230.0110.0430.4500.4730
PKR00000000000.0050.0120.1900.3690.4140.010
Table 8. Pairwise efficiency outranking indices for pairs of Polish voivodeships when considering inhabitants’ health improvement level.
Table 8. Pairwise efficiency outranking indices for pairs of Polish voivodeships when considering inhabitants’ health improvement level.
PEOIZPMPOMWMPDLLBUWLKPKPMAZLBLDSLOPOLDZSLSWMLPPKR
ZPM10.948110.97301111111111
POM0.0521110.89701111111111
WM0011000.9950.0700.9040.92110.85100.78611
PDL000100000.2400.01410.32000.2090.8440.767
LBU0.0270.10311101111110.987111
WLKP1111111111111111
KP000.005100100.5650.39510.64000.52410.996
MAZ000.930100110.9890.939110111
LBL000.0960.760000.4350.01110.27810.68800.4890.9621
DSL000.0790.986000.6050.0610.722110.72800.6350.9420.947
OPO0000000000100000.015
LDZ000.1490.680000.36000.3120.2721100.01111
SL00110.01301111111111
SW000.2140.791000.47600.5110.36510.9890111
MLP0000.15600000.0380.058100010.478
PKR0000.233000.004000.0530.9850000.5221
Table 9. Distribution of distances to the best unit for Polish voivodeships when considering financial management level.
Table 9. Distribution of distances to the best unit for Polish voivodeships when considering financial management level.
DAII[0.0, 0.1](0.1, 0.2](0.2, 0.3](0.3, 0.4](0.4, 0.5](0.5, 0.6](0.6, 0.7](0.7, 0.8](0.8, 0.9](0.9, 1.0]
ZPM0001000000
POM1000000000
WM000.7570.243000000
PDL000.4750.4830.04200000
LBU1000000000
WLKP00.9930.0070000000
KP00.1420.4530.3790.02600000
MAZ0000.8570.14300000
LBL0000.1390.7750.0860000
DSL00000.3120.6880000
OPO0000.2400.7080.0520000
LDZ000.7050.295000000
SL0000000.2200.7430.0370
SW000.3660.6280.00600000
MLP00.0430.9570000000
PKR00000.5450.4550000
Table 10. Distribution of ranks for Polish voivodeships when considering financial management level.
Table 10. Distribution of ranks for Polish voivodeships when considering financial management level.
ERAI12345678910111213141516
ZPM000000.0110.0460.1820.0620.6240.0680.0070000
POM0.3770.62300000000000000
WM0000.0480.3180.2070.0890.1430.1950000000
PDL00000.0010.2060.2000.2050.1330.1640.09100000
LBU0.6230.37700000000000000
WLKP000.9000.0930.00700000000000
KP000.1000.0880.2480.1690.0660.0770.0740.0610.11700000
MAZ00000000.0030.1510.0540.5630.1870.042000
LBL0000000000.0120.0460.3790.4240.13900
DSL00000000000000.3820.6180
OPO000000000.0050.0590.1140.3000.3390.0920.0910
LDZ0000.1000.1260.3650.3320.07700000000
SL0000000000000001
SW000000.0130.2670.3130.3800.0260.00100000
MLP0000.6710.3000.0290000000000
PKR000000000000.1270.1950.3870.2910
Table 11. Pairwise efficiency outranking indices for pairs of Polish voivodeships when considering financial management level.
Table 11. Pairwise efficiency outranking indices for pairs of Polish voivodeships when considering financial management level.
PEOIZPMPOMWMPDLLBUWLKPKPMAZLBLDSLOPOLDZSLSWMLPPKR
ZPM1000.285000.2440.9990.98610.9250.01210.08801
POM11110.40311111111111
WM1010.661000.51211110.51010.8160.0831
PDL0.71500.3391000.1970.8001110.25110.5460.0051
LBU10.59711111111111111
WLKP1011010.92011111110.9961
KP0.75600.4880.80300.08010.8131110.51710.6650.1721
MAZ0.001000.200000.18710.93210.80901001
LBL0.0140000000.068110.48001000.752
DSL00000000010.10201000.287
OPO0.0750000000.1910.5200.8981010.00600.754
LDZ0.98800.4900.749000.48311111110.1091
SL0000000000001000
SW0.91200.1840.454000.3351110.99401101
MLP100.9170.99500.0040.82811110.8911111
PKR000000000.2480.7130.24601001
Table 12. The voivodeships’ efficiency rankings imposed by four different measures at the level of health improvement: expected distance ( E d ), expected rank ( E R ), cross-efficiency ( C E ), and super-efficiency ( S E ).
Table 12. The voivodeships’ efficiency rankings imposed by four different measures at the level of health improvement: expected distance ( E d ), expected rank ( E R ), cross-efficiency ( C E ), and super-efficiency ( S E ).
ZPMPOMWMPDLLBUWLKPKPMAZLBLDSLOPOLDZSLSWMLPPKR
E d 0.1090.1520.4620.5820.1790.0000.5140.3720.5230.4920.7540.5480.3020.5260.6590.652
23713419610816125111514
E R 2.1103.0347.45612.5303.8601.0009.8856.13210.2159.16415.99011.3104.9969.77414.33914.205
23713411061181612591514
C E 0.1020.1550.4710.5980.1780.0000.5250.3780.5070.4950.7600.5260.2940.5090.6560.630
23713411169816125101514
S E 0.0650.1300.3830.5140.114-0.1400.4580.3240.4140.3740.7260.4700.2330.4690.5650.560
24813311069716125111514
Table 13. Kendall’s τ coefficient for ranking procedures at each hierarchy level.
Table 13. Kendall’s τ coefficient for ranking procedures at each hierarchy level.
Comprehensive AnalysisInhabitants’ Health ImprovementEffective Financial ManagementConsumer Satisfaction
Ed ER CE SE Ed ER CE SE Ed ER CE SE Ed ER CE SE
E d 1.001.000.730.721.000.970.970.951.000.970.970.951.001.000.950.90
E R 1.001.000.730.720.971.000.970.920.971.000.970.921.001.000.950.90
C E 0.730.731.000.850.970.971.000.950.970.971.000.920.950.951.000.92
S E 0.720.720.851.000.950.920.951.000.950.920.921.000.900.900.921.00
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Labijak-Kowalska, A.; Kadziński, M.; Mrozek, W. Robust Additive Value-Based Efficiency Analysis with a Hierarchical Structure of Inputs and Outputs. Appl. Sci. 2023, 13, 6406. https://doi.org/10.3390/app13116406

AMA Style

Labijak-Kowalska A, Kadziński M, Mrozek W. Robust Additive Value-Based Efficiency Analysis with a Hierarchical Structure of Inputs and Outputs. Applied Sciences. 2023; 13(11):6406. https://doi.org/10.3390/app13116406

Chicago/Turabian Style

Labijak-Kowalska, Anna, Miłosz Kadziński, and Weronika Mrozek. 2023. "Robust Additive Value-Based Efficiency Analysis with a Hierarchical Structure of Inputs and Outputs" Applied Sciences 13, no. 11: 6406. https://doi.org/10.3390/app13116406

APA Style

Labijak-Kowalska, A., Kadziński, M., & Mrozek, W. (2023). Robust Additive Value-Based Efficiency Analysis with a Hierarchical Structure of Inputs and Outputs. Applied Sciences, 13(11), 6406. https://doi.org/10.3390/app13116406

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics