# A Multicriteria Decision Aid-Based Model for Measuring the Efficiency of Business-Friendly Cities

^{1}

^{2}

^{3}

^{4}

^{5}

^{6}

^{7}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Literature Review

## 3. BFC Process Efficiency

- C1: A strategic approach in development planning.
- C2: Organizational capacity for support of the local economy.
- C3: Involvement of the economy in the work of the local government (economic council).
- C4: An effective system for issuing building permits.
- C5: Availability of information for investment.
- C6: Promotion of investment.
- C7: Creditworthiness and financial stability.
- C8: Promotion of employment and development.
- C9: Encouraging private–public partnerships.
- C10: Adequate infrastructure.
- C11: Transparent policy of taxes and incentives.
- C12: Application of information technologies.

_{i}(i = 1,…,12) [10,44,45,61]. It is given in Table 1 by NALED’s criterion validity rating.

## 4. Background

- -
- Choice (determining the “best” alternative);
- -
- Ranking (ranking the alternatives); and
- -
- Sorting (assigning alternatives to predefined and ordered classes).

#### 4.1. DEA

_{s}(DMUj, j = 1,…,n), which uses inputs x

_{ij}(i = 1,…,m) to produce outputs y

_{rj}(r = 1,…,s). The absolute efficiency measure model is as follows [86]:

_{i}(i = 1,…,m) are input multipliers and u

_{r}(r = 1,…,s) are output multipliers (weights). The above definition corresponds to a discrete MCDM. The determination of weights is a very sensitive and complicated process. The idea behind the DEA model is to avoid a priori weights determination. The authors of the DEA model in Charnes et al. [87] allowed each DMU to choose the most appropriate set of weights, with the goal of becoming as efficient as possible compared with the other units in the observing set. The linear programming (LP) weighted form of the basic constant return to scale model (DEA CCR or DEA CRS) with output orientation [87] is as follows:

_{k}are obtained by solving the linear model of Equations (1)–(5) n—times (once for each DMU with the goal of comparing it with other DMUs). As a solution of basic Charnes, Cooper, and Rhodes (CCR) DEA models [87], all efficient units are assessed with the even efficiency scores h

_{k}(k = 1,…,n) equal to 1 while the other inefficient ones are assessed with a score greater than 1 (it is usually calculated as the reciprocal value less than one). All inefficient units are enveloped by the production frontier, consisting of efficient DMUs. The efficient DMUs are composed of real-efficient or virtual-composite peer units(lying on the efficient frontier) for each of the inefficient DMUS. This model is transformed into the so-called Banker, Charnes and Cooper (BCC) model, which is described in [88], to incorporate the variable return to scale assumption. Namely, with respect to the DEA CRS model, the DEA BCC or DEA VRS model has an additional variable u* that defines the position of the auxiliary hyperplane lying above or at each DMU included in the analysis and checks that the specified DMU has reached the desired output level with minimum input engagement and that all possible overlapping hyperplanes of all DMUs are selected from the one that has the least horizontal distance from the observed DMU to this hyperplane. For u* = 0, the BCC model is reduced to the CCR model:

#### 4.2. SFA

#### 4.3. Classification

## 5. Methodology

- Fulfillment of the necessary condition for DEA application, i.e., strong relation between the number of input and outputs and DMUs. According to the literature [107], the general rule of thumb is as follows: ((number of inputs + number of outputs) × 3 <Number of DMUs. There are also milder conditions, set by authors in [107], which requires two DMUs for each input and output;
- Eliminating the noise in the data;
- Increases the readability of the results; and
- Speeds up the calculation.

#### 5.1. Basic Motivation for Integrating DEA and SFA with Classification

#### 5.2. Integration of the Classification and Efficiency Evaluation

## 6. Case Study: Evaluating of the Effectiveness of the BFC Process

#### 6.1. Data

_{ij}, i = 1,…12, j = 1,…20), while the amount of investment per capita is used as an output (y

_{j}, j = 1,…20). The case study of the efficiency evaluation of the BFC process of cities and local governments in the Republic of Serbia uses data, provided by NALED. The input and output criteria database together with BFC scores and ranking according to NALED’s methodology (normalized value of C

_{i}× w

_{i}, i = 1,…,12) are presented in Table 2.

#### 6.2. Efficiency Evaluation: Preliminary Results and Classification

## 7. Results Discussion

## 8. Conclusions

- -
- Enables proper use of the DEA methodology with appropriate degrees of freedom;
- -
- Reduces noise in the data; and
- -
- Provides better quality results as proved by naïve Bayes classification.

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Verbruggen, F.; Sutherland, J.; Werf, J.M.; Brinkkemper, S.; Sutherland, A. Process Efficiency—Adapting Flow to the Agile Improvement Effort. In Proceedings of the 52nd Hawaii International Conference on System Sciences (HICSS), Grand Wailea, Maui, 8–11 January 2019; pp. 1–7. [Google Scholar]
- Farell, M.J. The Measurement of Productive Efficiency. J. R.Stat. Soc. Ser. A Stat.Soc.
**1957**, 120, 253–290. [Google Scholar] [CrossRef] - Office of the Finanacial Managemnet, State of the Washington. Performance Measure Guide. Available online: http://www.ofm.wa.gov/budget/instructions/other/2009performancemeasureguide.pdf (accessed on 1 June 2020).
- Kaplan, R.S.; Norton, D.P. Translating Strategy into Action—The Balanced ScoreCard; Harward Bussines School Press: Boston, MA, USA, 1996. [Google Scholar]
- Dario, C.; Simar, L. Advaced Robust and Nonparametric Methods in Efficiency Analysis Methodology and Applications; Springer: New York, NY, USA, 2007; ISBN 987-0-387-35155-1. [Google Scholar]
- Coelli, T.J.; Rao, D.S.P.; O’Donnell, C.J.; Battese, G.E. An Introduction to Efficiency and Productivity Analysis; Springer: New York, NY, USA, 2005; ISBN 978-0-387-25895-9. [Google Scholar]
- Doumpos, M.; Zouponidis, C. Multicriteria Decision Aid Classification Methods (Applied Optimization); Springer: New York, NY, USA, 2002; ISBN 1402008058 (ISBN13: 9781402008054). [Google Scholar]
- Zopounidis, C.; Doumpos, M. Multicriteria classification and sorting methods: A literature review. Eur. J. Oper. Res.
**2002**, 138, 229–246. [Google Scholar] [CrossRef] - Yeveyeva, I. Solving Classification Problems with Multicriteria Decision Aiding Approaches. Ph.D. Thesis, Faculty of Information Technology, University of Jyvaskyla, Jyväskylä, Finland, 2007. [Google Scholar]
- Ranđelović, M.; Stanković, J.; Savić, G.; Kuk, K.; Ranđelović, D. An Approach to Determining the Importance of Criteria in the Process of Certifying a City as a Business-Friendly Environment. Interfaces
**2018**, 48, 156–165. [Google Scholar] [CrossRef] - NALED. Business Friendly Certification BFC SEE. Available online: https://naled.rs/certifikacija-opstina-bfc-see (accessed on 1 June 2020).
- Sarkis, J. A comparative analysis of DEA as a discrete alternative multiplecriteria decision tool. Eur. J. Oper. Res.
**2000**, 123, 543–557. [Google Scholar] [CrossRef] - Stewart, T.J. Relationships between data envelopment analysis and multicriteria decision analysis. J. Oper. Res. Soc.
**1996**, 47, 654–665. [Google Scholar] [CrossRef] - Prado, J.; Heijungs, R. Implementation of stochastic multi attribute analysis (SMAA) in comparative environmental assessments. Environ. Model. Softw.
**2018**, 109, 223–231. [Google Scholar] [CrossRef] - Ploskas, N.; Stiakakis, E.; Fouliras, P. Assessing Computer Network Efficiency Using Data Envelopment Analysis and Multicriteria Decision Analysis Techniques. J. Multi-Criteria Decis. Anal.
**2014**, 22. [Google Scholar] [CrossRef] - Bigaret, S.; Hodgett, R.E.; Meyer, P. Supporting the Multi-Criteria Decision Aiding process: R and the MCDA package. EURO J. Decis. Process.
**2017**, 5, 169–194. [Google Scholar] [CrossRef] [Green Version] - Brasil, A.T.; Pinheiro, R.P.; Coelho, L.V.A.; Costa, C.N. Comparison of Two MCDA Classification Methods over the Diagnosis of Alzheimer’s Disease. In Rough Sets and Knowledge Technology. RSKT 2009; Wen, P., Li, Y., Polkowski, L., Yao, Y., Tsumoto, S., Wang, G., Eds.; Springer: Berlin/Heidelberg, Germany, 2009; Volume 5589, pp. 334–341. [Google Scholar]
- Longaray, A.; Ensslin, L.; Ensslin, S.; Alves, G.; Dutra, A.; Munhoz, P. Using MCDA to evaluate the performance of the logistics process in public hospitals: The case of a Brazilian teaching hospital. Int. Trans. Oper. Res.
**2018**, 25, 133–156. [Google Scholar] [CrossRef] [Green Version] - Roy, J.; Adhikary, K.; Kar, S.; Pamucar, D. A rough strength relational DEMATEL model for analysing the key success factors of hospital service quality. Decis. Mak. Appl. Manag. Eng.
**2018**, 1, 121–142. [Google Scholar] [CrossRef] - Figueira, J.; Greco, S.; Ehrgott, M. (Eds.) Multiple Criteria Decision Analysis: State of the Art Surveys; International Series in Operations Research & Management Science; Springer: Boston, MA, USA, 2005. [Google Scholar]
- Lahdelma, R.; Salminen, P.; Hokkanen, J. Using Multicriteria Methods in Environmental Planning and Management. Environ. Manag.
**2014**, 26, 595–605. [Google Scholar] [CrossRef] [PubMed] - Coutinho-Rodrigues, J.; Simao, A.; Antunes, C.H. A gis-based multicriteriaspatial decision support system for planning urban infrastructures. Decis. Support Syst.
**2011**, 51, 720–726. [Google Scholar] [CrossRef] - Bagherikahvarin, M.; De Stem, Y. Using the A ranking method based on DEA and PROMETHEE II (a rank based on DEA & PR.II). Measurement
**2016**, 89, 333–342. [Google Scholar] - Botti, L.; Peypoch, N. Multi-criteria ELECTRE method and destination competitiveness. Tour. Manag. Perspect.
**2013**, 6, 108–113. [Google Scholar] [CrossRef] - Papamichail, K.N.; French, S. 25 years of MCDA in nuclear emergency management. IMA J. Manag. Math.
**2013**, 24, 481–503. [Google Scholar] [CrossRef] [Green Version] - Sawicka, H.; Węglinski, S.; Witort, P. Application of multiple criteria decision aid methods in logistic systems. LogForum
**2010**, 6, 3–10. [Google Scholar] - Biswas, P.; Pal, B.B. A fuzzy goal programming method to solve congestion management problem using genetic algorithm. Decis. Mak. Appl. Manag. Eng.
**2019**, 2, 36–53. [Google Scholar] [CrossRef] - Belacel, N. Multicriteria assignment method PROAFTN: Methodology and medical applications. Eur. J. Oper. Res.
**2000**, 125, 175–183. [Google Scholar] [CrossRef] - Brasil, A.T.; Pinheiro, P.R.; Coelho, A.L.V. The impact of the prototype selectionon a multicriteria decision aid classification algorithm. In Innovations and Advances in Computer Sciences and Engineering; Springer: New York, NY, USA, 2010; Volume 1, pp. 379–382. [Google Scholar]
- Jacobs, R.; Smith, P.; Street, A. Measuring Efficiency in Health Care; Cambridge University Press: Cambridge, UK, 2006. [Google Scholar]
- Berger, A.N.; Hunter, W.C.; Timme, S.G. The Efficiency of Financial Institutions: AReview and Preview of Research Past, Present, and Future. J. Bank. Financ.
**1993**, 17, 221–249. [Google Scholar] [CrossRef] - Sarkis, J.; Talluri, S. Eco-efficiency measurement using data envelopment analysis: Research data practitioner issues. Integr. Environ. Assess. Manag.
**2004**, 6, 91–123. [Google Scholar] [CrossRef] - Carlsson, B. The Measurement of Efficiency in Production: An Application to Swedish Manufacturing Industries 1968. Swedish J. Econ.
**1972**, 74, 468–485. [Google Scholar] [CrossRef] - Klimchik, A.; Ambiehl, A.; Garnier, S.; Furet, B.; Pashkevich, A. Efficiency evaluation of robots in machining applications using industrial performance measure. Robot. Comput.-Integr Manuf.
**2017**, 48, 12–29. [Google Scholar] [CrossRef] - Peypoch, N.; Solonandrasana, B. Research note: Technical efficiency in the tourism industry. Tour. Econ.
**2006**, 12, 653–657. [Google Scholar] [CrossRef] - Haar, S.; Segers, R.M.; Jehn, K. Measuring the effectiveness of emergency management teams: Scale development and validation. Int. J. Emerg. Manag.
**2013**, 9, 258. [Google Scholar] [CrossRef] - Andrejić, M.; Kilibarda, M. The problems of measuring efficiency in logistics. Vojnotehnicki glasnik
**2013**, 61. [Google Scholar] [CrossRef] [Green Version] - Filippini, M.; Farsi, M. Regulation and Measuring Cost-Efficiency with Panel Data Models: Application to Electricity Distribution Utilities. Rev. Ind. Organ.
**2004**, 25, 1–19. [Google Scholar] - Ferrera, J.; Pedraja-Chaparro, F.; Salinas-Jimenez, J. Measuring Efficiency in Education: An Analysis of Different Approaches for Incorporating Non-discretionary Inputs. Appl. Econ.
**2008**, 40, 1323–1339. [Google Scholar] [CrossRef] [Green Version] - Liket, K.C.; Maas, K. Nonprofit organizational effectiveness: Analysis of best practices. Nonprofit Volunt. Sect. Q.
**2015**, 44, 268–296. [Google Scholar] [CrossRef] - BFC SEE Network. Available online: http://bfc-see.org/about-bfc-see-network (accessed on 1 June 2020).
- Radukić, S.; Stanković, J. Evaluation of Local Business Environment in the Republic of Serbia. ProcediaEcon. Financ.
**2015**, 19, 353–363. [Google Scholar] [CrossRef] [Green Version] - Stojanović, B.; Stanković, J.; Ranđelović, M. The City of Niš Competitiveness Analysis in the Field of Foreign Direct Investment. Econ. Enterp.
**2012**, 60, 167–178. [Google Scholar] [CrossRef] [Green Version] - Ranđelović, D.; Stanković, J.; Anđelković, M.; Ranđelović, M. Application of AHP method in cities certification process. Management
**2014**, 69, 75–84. [Google Scholar] - Ranđelović, M.; Stanković, J.; Ranđelović, D.; Suknović, M. Methods for determination weights in multi-criteria model of cities certification methods for determination weights in multi-criteria model of cities certification. Tech. Technol. Educ. Manag.
**2013**, 8, 1404–1412. [Google Scholar] - Randjelović, D.; Janjić, J.; Stanković, J.; Randjelović, M.; Pešić, M. Weights determination in MCDM model combining the techniques of mathematical and statistical analysis. Metal. Int.
**2013**, XVIII, 217–221. [Google Scholar] - Radulovic, B.; Dragutinovic, S. Efficiency of local self-governments in Serbia: An SFA approach. Industrija
**2015**, 43, 123–142. [Google Scholar] [CrossRef] [Green Version] - Afonso, A.; Fernandes, S. Assessing and Explaining the Relative Efficiency of Local Government. J. Socio-Econ.
**2008**, 37, 1946–1979. [Google Scholar] [CrossRef] - Balaguer, M.T.; Prior, D.; Tortosa, E. On the determinants of local government performance: A two-stage nonparametric approach. Eur. Econ. Rev.
**2007**, 51, 425–451. [Google Scholar] [CrossRef] [Green Version] - Sousa, M.D.C.S.D.; Stošić, B. Technical Efficiency of the Brazilian Municipalities: Correcting Nonparametric Frontier Measurements for Outliers. J. Product. Anal.
**2005**, 24, 157–181. [Google Scholar] [CrossRef] - De Borger, B.; Kerstens, K. Cost Efficiency of Belgian Local Governments: A Comparative analysis of FDH, DEA, and econometric approaches. Reg. Sci. Urban Econ.
**1996**, 26, 145–170. [Google Scholar] [CrossRef] - Geys, B.; Heinemann, F.; Kalb, A. Voter involvement, fiscal autonomy and public sector efficiency: Evidence from German municipalities. Eur. J. Political Econ.
**2010**, 26, 265–278. [Google Scholar] [CrossRef] [Green Version] - Lo Storto, C. Evaluating technical efficiency of Italian major municipalities: A data envelopment analysis model. Procedia Soc. Behav. Sci.
**2013**, 81, 346–350. [Google Scholar] [CrossRef] [Green Version] - Worthington, A.C. Cost Efficiency in Australian Local Government: A Comparative Analysis of Mathematical Programming and Econometric Approaches. Financ. Account. Manag.
**2000**, 16, 201–223. [Google Scholar] [CrossRef] [Green Version] - Westhuizen, G.; Dollery, B. South African Local Government Efficiency Measurement; Working Paper Series 4.1-19; Centre for Local Government, School of Business, Economics and Public Policy, University of New England: Armidale, Australia, 2009. [Google Scholar]
- Ray, S. Data Envelopment Analysis; Cambridge University Press: Cambridge, UK, 2004; ISBN 0-521-80256-3. [Google Scholar]
- Charnes, A.; Cooper, W.; Lewin, A.Y.; Seiford, L.M. Data Envelopment Analysis Theory, Methodology and Applications. J. Oper. Res. Soc.
**1997**, 48, 332–333. [Google Scholar] [CrossRef] - Cooper, W.W.; Seiford, L.M.; Zhu, J. Data Envelopment Analysis, Handbook on Data Envelopment Analysis; Springer: Boston, MA, USA, 2004; ISBN 978-1-4020-7798-2. [Google Scholar]
- Kongar, E.; Pallis, J.; Sobh, T. Non-parametric Approach for Evaluating the Performance of Engineering Schools. Int. J. Eng. Educ.
**2010**, 26, 1210–1219. [Google Scholar] - Ji, Y.; Lee, C. Data envelopment analysis. Stata J.
**2010**, 2, 267–280. [Google Scholar] [CrossRef] [Green Version] - Stanković, J.; Radenković;-Jocić, D. The role of business environment in the promotion of investment activities: Case study and municipalities in the Republic of Serbia. Teme
**2017**, XLI, 457–473. [Google Scholar] - Beiragh, R.; Alizadeh, R.; Shafiei Kaleibari, S.; Cavallaro, F.; Zolfani, S.; Bausys, R.; Mardani, A. An integrated Multi-Criteria Decision Making Model for Sustainability Performance Assessment for Insurance Companies. Sustainability
**2020**, 12, 789. [Google Scholar] [CrossRef] [Green Version] - Kumbhakar, S.; Parmeter, C.; Zelenyuk, V. Stochastic Frontier Analysis: Foundations and Advances; Working Papers 2017-10; Department of Economics, University of Miami: Coral Gables, FL, USA, 2017. [Google Scholar]
- Seale, J., Jr. Stochastic Frontier Analysis. J. Econ. Lit.
**2000**, 38, 936–938. [Google Scholar] - Battese, G.; Corra, G. Estimation of a production frontier model: With application to the pastoral zone of eastern Australia. Aust. J. Agric. Resour. Econ.
**1977**, 21, 169–179. [Google Scholar] [CrossRef] [Green Version] - Belotti, F.; Daidone, S.; Ilardi, G.; Atella, V. Stochastic Frontier Analysis Using Stata. Stata J.
**2012**, 13, 719–758. [Google Scholar] [CrossRef] [Green Version] - Chitnis, A.; Vaidya, O. Efficiency ranking method using SFA and TOPSIS (ERM-ST): Case of Indian banks. Benchmarking Int. J. Res.
**2018**, 25, 471–488. [Google Scholar] [CrossRef] - Johnson, A.; Kuosmanen, T. An Introduction to CNLS and StoNED Methods for Efficiency Analysis: Economic Insights and Computational Aspects and Formulations. In Benchmarking for Performance Evaluation: A Production Frontier Approach; Ray, S., Kumbhakar, S., Dua, P., Eds.; Springer: Berlin/Heidelberg, Germany, 2014. [Google Scholar]
- Thoraneenitiyan, N.; Avkiran, K.N. Measuring the impact of restructuring and country-specific factors on the efficiency of post-crisis East Asian banking systems:Integrating DEA with SFA. Socio.-Econ. Plan. Sci.
**2009**, 43, 240–252. [Google Scholar] [CrossRef] - Bezat-Jarzebowska, A. A concept of technical efficiency measurement based on the integrated use of the SFA and DEA methods. Sci. Works Wrocław University Econ.
**2012**, 261, 11–24. [Google Scholar] - François-Seck, F.; Al-Mouksit, A.; Wassongma, H. DEA and SFA research on the efficiency of microfinance institutions: A meta-analysis. World Dev.
**2018**, 107, 176–188. [Google Scholar] - Bogetoft, P.; Otto, L. Stochastic Frontier Analysis SFA. In Benchmarking with DEA, SFA, and R; Operations Research Management Science Series 157; Springer: Berlin/Heidelberg, Germany, 2011; pp. 197–231. [Google Scholar] [CrossRef]
- Azadeh, A.; Ghaderi, S.F.; Omrani, H.; Eivazy, H. An integrated DEA-COLS-SFA algorithm for optimization and policy making of electricity distribution units. Energy Policy
**2009**, 37, 2605–2618. [Google Scholar] [CrossRef] - Anouze, A.L.; Bou-Hamad, I. Data envelopment analysis and data mining to efficiency estimation and evaluation. Int. J. Islamic Middle East. Financ. Manag.
**2019**, 12, 169–190. [Google Scholar] [CrossRef] [Green Version] - Azadeh, A.; Saberi, M.; Moghaddam, R.; Javanmardi, L. An integrated data envelopment analysis-artificial neural network-rough set algorithm for assessment of personnel efficiency. Expert Syst. Appl.
**2011**, 38, 1364–1373. [Google Scholar] [CrossRef] - Antonuccia, L.; Crocettaa, C.; D’Ovidio, F. Evaluation of Italian Judicial System. Procedia Econ. Financ.
**2014**, 17, 121–130. [Google Scholar] [CrossRef] [Green Version] - Fethi, M.; Pasiouras, F. Assessing bank efficiency and performance with operational research and artificial intelligence techniques: A survey. Eur. J. Oper. Res.
**2010**, 204, 189–198. [Google Scholar] [CrossRef] - Swinburn, G.; Goga, S.; Murphy, F. Local Economic Development: A Primer Developing and Implementing Local Economic Development Strategies and Action Plan; The World Bank: Washington, DC, USA, 2006; Available online: http://siteresources.worldbank.org/INTLED/423069-1099670772921/20738133/led_primer.pdf (accessed on 1 June 2020).
- Ranđelović, M.; Nedeljković, S.; Jovanović, M.; Čabarkapa, M.; Stojanović, V.; Aleksić, A.; Ranđelović, D. Use of Determination of the Importance of Criteria in Business-Friendly Certification of Cities as Sustainable Local Economic Development Planning Tool. Symmetry
**2020**, 12, 425. [Google Scholar] [CrossRef] [Green Version] - Keeney, R.; Raiffa, H. Decisions with Multiple Objectives: Preferences and Value Tradeoffs; J. Wiley: New York, NY, USA, 1976. [Google Scholar]
- Koksalan, M.; Wallenius, J.; Zionts, S. Multiple Criteria Decision Making: _From Early History to 21th Century; World Scientific: Singapore, 2011; ISBN 978-981-4335-58-4. [Google Scholar]
- Ishizaka, A.; Nemery, P. Multi-Criteria Decision Analysis: Methods and Software; J. Wiley: New York, NY, USA, 2013. [Google Scholar]
- Tsoukiàs, A. From decision theory to decision aiding methodology. Eur. J. Oper. Res.
**2008**, 187, 138–161. [Google Scholar] [CrossRef] [Green Version] - Belton, V.; Vickers, S.P. Demystifying DEA—A visual interactive approach based on multi criteria analysis. J. Oper. Res. Soc.
**1993**, 44, 883–896. [Google Scholar] - Cooper, W.W.; Seiford, L.M.; Tone, K. Data Envelopment Analysis: A Comprehensive Text with Models, Applications, References and DEA-Solver Software; Kluwer Academic Publishers: Boston, MA, USA, 2000. [Google Scholar]
- Podinovski, V.V. Side effects of absolute weight bounds in DEA models. Eur. J. Oper. Res.
**1999**, 115, 583–595. [Google Scholar] [CrossRef] - Charnes, A.; Cooper, W.W.; Rhodes, E. Measuring the efficiency of decision making units. Eur. J. Oper. Res.
**1978**, 2, 429–444. [Google Scholar] [CrossRef] - Banker, R.D.; Charnes, A.; Cooper, W.W. Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis. Manag. Sci.
**1984**, 30, 1078–1092. [Google Scholar] [CrossRef] [Green Version] - Aigner, D.; Lovell, C.A.K.; Schmidt, P. Formulation and estimation of stochastic frontier production function models. J. Econ.
**1977**, 6, 21–37. [Google Scholar] [CrossRef] - Meeusen, W.; Van Den Broeck, J. Efficiency estimation from Cobb–Douglas production functions with composed error. Int.Econ. Rev.
**1977**, 18, 435–444. [Google Scholar] [CrossRef] - Kumbhakar, S.C.; Lovell, C.A.K. Stochastic Frontier Analysis; Cambridge University Press: Cambridge, UK, 2000. [Google Scholar]
- MacKay, D. Information Theory, Inference, and Learning Algorithms; CambridgeUniversity Press: Cambridge, UK, 2003. [Google Scholar]
- Mitchell, T. Machine Learning; McGraw-Hill Science/Engineering/Math: New York, NY, USA, 1997. [Google Scholar]
- Haykin, S. Neural Networks. A Comprehensive Foundation, 2nd ed.; Prentice Hall: Upper Saddle River, NJ, USA, 2005. [Google Scholar]
- Russell, S.; Norvig, P. Artificial Intelligence: A Modern Approach; Prentice Hall: Upper Saddle River, NJ, USA, 2003. [Google Scholar]
- Langley, P.; Iba, W.; Thompson, K. An analysis of Bayesian classifiers. In Proceedings of the Tenth National Conference on Artificial Intelligence, San Jose, CA, USA, 12–16 July 1992; pp. 399–406. [Google Scholar]
- Friedman, N.; Geiger, D.; Goldszmidt, M. Bayesian network classifiers. Mach. Learn.
**1997**, 29, 131–163. [Google Scholar] [CrossRef] [Green Version] - Almuallim, H.; Dietterich, T.G. Learning with many irrelevant features. In Proceedings of the Ninth National Conference on Artificial Intelligence, Anaheim, CA, USA, 14–19 July 1991; pp. 547–552. [Google Scholar]
- Weka (University of Waikato: New Zealand). Available online: http://www.cs.waikato.ac.nz/ml/weka (accessed on 1 June 2020).
- Liu, H.; Motoda, H. Feature Selection for Knowlegde Discovery and Data Mining; Kluwer Academic Publishers: London, UK, 1998. [Google Scholar]
- Abe, N.; Kudo, M. Entropy criterion for classifier-independent feature selection. Lect. Notes Comput. Sci.
**2005**, 3684, 689–695. [Google Scholar] - Hall, M.A.; Smith, L.A. Practical feature subset selection for machine learning. In Proceedings of the 21st Australian Computer Science Conference (ACSC’98), Perth, Australia, 4–6 February1998; pp. 181–191. [Google Scholar]
- Moriwal, R.; Prakash, V. An efficient info-gain algorithm for finding frequent sequential traversal patterns from web logs based on dynamic weight constraint. In Proceedings of the CUBE International Information Technology Conference (CUBE’12), Pune, India, 3–5 September2012; pp. 718–723. [Google Scholar]
- Quinlan, J.R. C4.5: Programs for Machine Learning; Morgan Kaufmann Publishers: San Francisco, CA, USA, 1993. [Google Scholar]
- Thakur, D.; Markandaiah, N.; Raj, D.S. Re optimization of ID3 and C4.5 decision tree. Proceedingsof the International Conference on Computer and Communication Technology (ICCCT), Allahabad, India, 17–19 September 2010; pp. 448–450. [Google Scholar]
- Kononenko, I. Estimating attributes: Analysis and extensions of relief. In Machine Learning: ECML-94; Springer: Berlin/Heidelberg, Germany, 1994; pp. 171–182. [Google Scholar]
- Cooper, W.; Seiford, L.; Tone, K. Data Envelopment Analysis; Springer: New York, NY, USA, 2000. [Google Scholar]
- Urbanowicz, R.; Meeker, M.; La Cava, W.; Olson, R.; Moore, J. Relief-based feature selection: Introduction and review. J. Biomed. Inform.
**2018**, 85, 189–203. [Google Scholar] [CrossRef] - Powers, D. Evaluation: From precision, recall and F-measure to ROC, informedness, markedness & correlation. J. Mach. Learn. Technol.
**2011**, 2, 2229–3981. [Google Scholar] [CrossRef] - Domingos, P.; Pazzani, M. On the optimality of the simple Bayesian classifier under zero-one loss. Mach. Learn.
**1997**, 29, 103–130. [Google Scholar] [CrossRef] - Silva, T.C.; Tabak, B.M.; Cajueiro, D.O.; Dias, M.V.B. A comparison of DEA and SFA using micro-and macro-level perspectives: Efficiency of Chinese local banks. Phys. A
**2017**, 469, 216–223. [Google Scholar] [CrossRef] - Popović, M.; Savić, G.; Kuzmanović, M.; Martić, M. Using Data Envelopment Analysis and Multi-Criteria Decision-Making Methods to Evaluate Teacher Performance in Higher Education. Symmetry
**2020**, 12, 563. [Google Scholar] - Dong, Y.; Hamilton, R.; Tippett, M. Cost efficiency of the Chinese banking sector: A comparison of stochastic frontier analysis and data envelopment analysis. Econ. Model.
**2014**, 36, 298–308. [Google Scholar] [CrossRef] - Shumaiza, A.M.; Alcantud, J.C.R. An m-Polar Fuzzy PROMETHEE Approach for AHP-Assisted Group Decision-Making. Math. Comput. Appl.
**2020**, 25, 26. [Google Scholar] - Đalić, I.; Stević, Ž.; Karamasa, C.; Puška, A. A novel integrated fuzzy PIPRECIA—Interval rough SAW model: Green supplier selection. Decis. Mak. Appl. Manag. Eng.
**2020**, 3, 126–145. [Google Scholar] [CrossRef] - Pamučar, D. Fuzzy-dea model for measuring the efficiency of the transport quality. Mil. Tech. Cour.
**2011**, LIX, 40–61. [Google Scholar] [CrossRef] [Green Version] - Stojić, G.; Sremac, S.; Vasiljković, I. A fuzzy model for determining the justifiability of investing in a road freight vehicle fleet. Oper. Res. Eng. Sci. Theory Appl.
**2018**, 1, 62–75. [Google Scholar] [CrossRef] - Liu, F.; Aiwu, G.; Lukovac, V.; Vukic, M. A multicriteria model for the selection of the transport service provider: A single valued neutrosophic DEMATEL multicriteria model. Decis. Mak. Appl. Manag. Eng.
**2018**, 1, 121–130. [Google Scholar] [CrossRef] - Kevkić, T.S.; Odalović, M.T.; Petković, D.M. A stochastic model of gamma-ray induced oxide charge distribution and threshold voltage shift of MOS transistors. Nucl. Technol. Radiat.
**2012**, 27, 33–39. [Google Scholar] [CrossRef] - Kevkić, T.; Stojanović, V.; Joksimović, D. Application of the Generalized Logistic Functions in the Surface Potential Based MOSFET Modeling. J. Comput. Electron.
**2017**, 16, 90–97. [Google Scholar] [CrossRef]

C_{1} | C_{2} | C_{3} | C_{4} | C_{5} | C_{6} | C_{7} | C_{8} | C_{9} | C_{10} | C_{11} | C_{12} | |
---|---|---|---|---|---|---|---|---|---|---|---|---|

Evaluation of the criteria by Naled’s experts (w_{i}) | 1.250 | 0.900 | 0.670 | 1.190 | 0.660 | 0.710 | 1.000 | 0.750 | 1.080 | 1.210 | 1.500 | 0.830 |

Inputs | Output | NALED’s Evaluation | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

C_{1} | C_{2} | C_{3} | C_{4} | C_{5} | C_{6} | C_{7} | C_{8} | C_{9} | C_{10} | C_{11} | C_{12} | Investment | Score | Ranking | |

Municipality 1 | 0.80 | 1.06 | 1.00 | 0.73 | 0.88 | 1.00 | 1.00 | 0.73 | 0.64 | 0.83 | 1.00 | 1.00 | 520.02 | 0.88 | 10 |

Municipality 2 | 1.00 | 0.82 | 0.75 | 1.00 | 0.93 | 1.00 | 1.00 | 0.93 | 1.00 | 0.88 | 1.00 | 1.00 | 686.57 | 0.95 | 3 |

Municipality 3 | 0.63 | 0.95 | 0.80 | 0.94 | 0.86 | 1.00 | 0.90 | 0.75 | 0.67 | 0.94 | 0.93 | 1.00 | 580.64 | 0.86 | 14 |

Municipality 4 | 0.90 | 0.82 | 0.88 | 1.00 | 0.95 | 1.00 | 1.00 | 0.70 | 0.68 | 0.76 | 1.00 | 0.75 | 464.16 | 0.87 | 12 |

Municipality 5 | 1.00 | 0.62 | 1.00 | 0.78 | 0.60 | 0.67 | 1.00 | 0.60 | 0.59 | 0.98 | 0.83 | 1.00 | 315.94 | 0.82 | 19 |

Municipality 6 | 1.00 | 1.06 | 0.75 | 0.94 | 0.90 | 0.94 | 1.00 | 0.87 | 0.91 | 0.79 | 1.00 | 1.00 | 942.36 | 0.94 | 4 |

Municipality 7 | 1.00 | 0.94 | 1.00 | 0.78 | 0.70 | 0.78 | 1.00 | 0.57 | 0.73 | 0.70 | 1.00 | 0.50 | 879.20 | 0.82 | 18 |

Municipality 8 | 1.00 | 0.82 | 1.00 | 0.89 | 1.00 | 1.00 | 1.00 | 0.83 | 0.55 | 0.88 | 1.00 | 1.00 | 415.97 | 0.91 | 7 |

Municipality 9 | 1.00 | 0.82 | 1.00 | 0.67 | 0.65 | 1.00 | 1.00 | 0.87 | 0.96 | 0.81 | 0.83 | 1.00 | 622.95 | 0.88 | 11 |

Municipality 10 | 1.00 | 0.94 | 0.75 | 0.81 | 0.63 | 0.94 | 1.00 | 0.67 | 0.91 | 0.79 | 1.00 | 1.00 | 754.09 | 0.89 | 8 |

Municipality 11 | 1.00 | 0.77 | 0.75 | 0.83 | 0.73 | 1.00 | 1.00 | 0.53 | 0.64 | 0.76 | 0.83 | 1.00 | 687.33 | 0.83 | 17 |

Municipality 12 | 0.80 | 1.00 | 0.75 | 0.89 | 0.90 | 1.00 | 1.00 | 0.53 | 0.73 | 0.73 | 0.83 | 0.75 | 200.01 | 0.83 | 16 |

Municipality 13 | 0.80 | 1.00 | 1.00 | 0.74 | 0.73 | 1.00 | 1.00 | 0.77 | 0.46 | 0.77 | 1.00 | 1.00 | 111.78 | 0.85 | 15 |

Municipality 14 | 1.00 | 0.94 | 1.00 | 0.87 | 0.73 | 1.00 | 1.00 | 0.83 | 0.55 | 0.68 | 1.00 | 0.88 | 368.21 | 0.87 | 13 |

Municipality 15 | 1.00 | 0.94 | 1.00 | 1.00 | 0.90 | 1.00 | 1.00 | 1.00 | 1.00 | 0.94 | 1.00 | 0.62 | 995.82 | 0.96 | 2 |

Municipality 16 | 1.00 | 0.82 | 1.00 | 0.89 | 0.78 | 1.00 | 1.00 | 0.80 | 0.91 | 0.78 | 1.00 | 1.00 | 208.68 | 0.92 | 5 |

Municipality 17 | 1.00 | 0.82 | 1.00 | 0.78 | 0.88 | 0.89 | 1.00 | 0.67 | 0.55 | 0.83 | 0.67 | 0.75 | 306.58 | 0.81 | 20 |

Municipality 18 | 1.00 | 0.88 | 1.00 | 1.02 | 0.90 | 0.94 | 1.00 | 0.87 | 1.09 | 0.93 | 1.00 | 1.00 | 295.83 | 0.98 | 1 |

Municipality 19 | 1.00 | 0.95 | 0.60 | 0.97 | 0.93 | 1.00 | 1.00 | 0.85 | 0.58 | 0.94 | 1.00 | 1.00 | 432.21 | 0.91 | 6 |

Municipality 20 | 1.00 | 0.77 | 0.88 | 0.81 | 0.80 | 1.00 | 1.00 | 0.73 | 0.82 | 0.77 | 1.00 | 1.00 | 697.12 | 0.89 | 9 |

Max | 1 | 1.06 | 1 | 1.024 | 1 | 1 | 1 | 1 | 1.09 | 0.98 | 1 | 1 | 995.82 | 0.98 | |

Min | 0.63 | 0.62 | 0.60 | 0.67 | 0.60 | 0.67 | 0.90 | 0.53 | 0.46 | 0.68 | 0.67 | 0.50 | 111.78 | 0.81 | |

Average | 0.95 | 0.89 | 0.90 | 0.87 | 0.82 | 0.96 | 1.00 | 0.76 | 0.75 | 0.82 | 0.95 | 0.91 | 524.27 | 0.88 | |

SD | 0.10 | 0.11 | 0.13 | 0.10 | 0.11 | 0.09 | 0.02 | 0.13 | 0.18 | 0.08 | 0.09 | 0.15 | 248.77 | 0.05 |

Municipality | DEA CRS | DEA VRS | SFA | |||
---|---|---|---|---|---|---|

Score | Rank | Score | Rank | SFA | Rank | |

Municipality 1 | 0.716 | 10 | 1.000 | 1 | 1.000 | 1 |

Municipality 2 | 0.845 | 8 | 0.929 | 18 | 0.878 | 10 |

Municipality 3 | 0.933 | 6 | 1.000 | 1 | 1.000 | 1 |

Municipality 4 | 0.588 | 13 | 0.999 | 15 | 1.000 | 1 |

Municipality 5 | 0.505 | 15 | 1.000 | 1 | 1.000 | 1 |

Municipality 6 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 |

Municipality 7 | 1.000 | 1 | 1.000 | 1 | 0.582 | 15 |

Municipality 8 | 0.631 | 12 | 1.000 | 1 | 0.649 | 14 |

Municipality 9 | 0.838 | 9 | 1.000 | 1 | 0.883 | 9 |

Municipality 10 | 1.000 | 1 | 1.000 | 1 | 0.676 | 12 |

Municipality 11 | 0.960 | 5 | 1.000 | 1 | 1.000 | 1 |

Municipality 12 | 0.277 | 18 | 0.999 | 17 | 0.419 | 19 |

Municipality 13 | 0.203 | 20 | 0.999 | 15 | 0.505 | 16 |

Municipality 14 | 0.559 | 14 | 1.000 | 1 | 1.000 | 1 |

Municipality 15 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 |

Municipality 16 | 0.245 | 19 | 0.269 | 20 | 0.358 | 20 |

Municipality 17 | 0.501 | 16 | 1.000 | 1 | 0.658 | 13 |

Municipality 18 | 0.324 | 17 | 0.339 | 19 | 0.451 | 17 |

Municipality 19 | 0.674 | 11 | 1.000 | 1 | 0.425 | 18 |

Municipality 20 | 0.885 | 7 | 1.000 | 1 | 0.759 | 11 |

Average | 0.684 | 0.927 | 0.762 | |||

Max | 1.000 | 1 | 1.000 | |||

Min | 0.203 | 0.270 | 0.358 | |||

St Dev | 0.275 | 0.011 | 0.241 |

TP Rate | FP Rate | Precision | Recall | F-measure | ROC Area | Class | |
---|---|---|---|---|---|---|---|

0.875 | 1 | 0.778 | 0.875 | 0.824 | 0.438 | No | |

0 | 0.125 | 0 | 0 | 0 | 0.438 | Yes | |

Weighted Avg. | 0.7 | 0.825 | 0.622 | 0.7 | 0.659 | 0.438 |

Value | Rank | Attribute |
---|---|---|

0.1783 | 1 | C12 |

0.1070 | 2 | C6 |

0.0601 | 3 | C2 |

0.0505 | 4 | C8 |

0.0500 | 5 | C9 |

0.0019 | 6 | C3 |

0.0017 | 7 | C5 |

0.0016 | 8 | C10 |

−0.0003 | 9 | C11 |

−0.01 | 10 | C7 |

−0.015 | 11 | C1 |

−0.022 | 12 | C4 |

Municipality | DEA CRS | DEA VRS | SFA | |||
---|---|---|---|---|---|---|

Score | Rank | Score | Rank | SFA | Rank | |

Municipality 1 | 0.676 | 10 | 0.757 | 17 | 0.795 | 10 |

Municipality 2 | 0.787 | 7 | 0.861 | 12 | 0.751 | 11 |

Municipality 3 | 0.720 | 8 | 0.771 | 16 | 0.897 | 6 |

Municipality 4 | 0.588 | 13 | 0.652 | 18 | 0.687 | 15 |

Municipality 5 | 0.505 | 15 | 1.000 | 1 | 0.743 | 12 |

Municipality 6 | 0.919 | 4 | 0.985 | 10 | 1.000 | 1 |

Municipality 7 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 |

Municipality 8 | 0.631 | 11 | 1.000 | 1 | 1.000 | 1 |

Municipality 9 | 0.716 | 9 | 0.782 | 15 | 0.691 | 14 |

Municipality 10 | 0.832 | 6 | 0.832 | 13 | 0.707 | 13 |

Municipality 11 | 0.937 | 3 | 1.000 | 1 | 1.000 | 1 |

Municipality 12 | 0.242 | 19 | 0.999 | 7 | 0.199 | 20 |

Municipality 13 | 0.203 | 20 | 0.999 | 7 | 0.295 | 17 |

Municipality 14 | 0.559 | 14 | 0.974 | 11 | 0.808 | 9 |

Municipality 15 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 |

Municipality 16 | 0.245 | 18 | 0.262 | 20 | 0.234 | 19 |

Municipality 17 | 0.465 | 16 | 0.999 | 7 | 0.635 | 16 |

Municipality 18 | 0.324 | 17 | 0.34 | 19 | 0.267 | 18 |

Municipality 19 | 0.613 | 12 | 0.817 | 14 | 0.886 | 8 |

Municipality 20 | 0.885 | 5 | 1 | 1 | 0.890 | 7 |

Average | 0.642 | 0.852 | 0.724 | |||

Max | 1.000 | 1 | 1.000 | |||

Min | 0.203 | 0.262 | 0.199 | |||

St Dev | 0.254 | 0.218 | 0.271 |

TP Rate | FP Rate | Precision | Recall | F-Measure | ROC Area | Class | |
---|---|---|---|---|---|---|---|

1 | 1 | 0.9 | 1 | 0.947 | 0.444 | No | |

0 | 0 | 0 | 0 | 0 | 0.056 | Yes | |

Weighted Avg. | 0.9 | 0.9 | 0.81 | 0.9 | 0.853 | 0.406 |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Jovanović, M.; Nedeljković, S.; Ranđelović, M.; Savić, G.; Stojanović, V.; Stojanović, V.; Ranđelović, D.
A Multicriteria Decision Aid-Based Model for Measuring the Efficiency of Business-Friendly Cities. *Symmetry* **2020**, *12*, 1025.
https://doi.org/10.3390/sym12061025

**AMA Style**

Jovanović M, Nedeljković S, Ranđelović M, Savić G, Stojanović V, Stojanović V, Ranđelović D.
A Multicriteria Decision Aid-Based Model for Measuring the Efficiency of Business-Friendly Cities. *Symmetry*. 2020; 12(6):1025.
https://doi.org/10.3390/sym12061025

**Chicago/Turabian Style**

Jovanović, Mihailo, Slobodan Nedeljković, Milan Ranđelović, Gordana Savić, Vladica Stojanović, Vladimir Stojanović, and Dragan Ranđelović.
2020. "A Multicriteria Decision Aid-Based Model for Measuring the Efficiency of Business-Friendly Cities" *Symmetry* 12, no. 6: 1025.
https://doi.org/10.3390/sym12061025