# Methodical Aspects of MCDM Based E-Commerce Recommender System

^{1}

^{2}

^{3}

^{*}

*J. Theor. Appl. Electron. Commer. Res.*

**2021**,

*16*(6), 2192-2229; https://doi.org/10.3390/jtaer16060122

## Abstract

**:**

## 1. Introduction

#### 1.1. Nature, Significance, and Commonness of the Mobile Phone Selection Problem

- Product price—gross price of the phone.
- Random Access Memory (RAM) size—RAM size is a parameter of mobile phones that determines the capacity to install various applications on them.
- Processor (CPU) type—the processor is a powerful built-in device in smartphones that provides flexible access to applications. Almost all mobile phones have different processors by speed and type. A higher number of cores and higher processor speed are the preferred features.
- Built-in memory size—internal memory is a mobile phone feature that allows storing data on it with a defined capacity.
- The camera quality—the camera in a smartphone is used to take pictures and videos. The camera is responsible for resolution image quality.
- Screen dimension.
- Battery capacity—duration of a rechargeable battery of a product.
- Weight—body’s relative mass.

- Operating System (OS). Provides user interface and interaction functions. Smartphones run on different operating systems: Android (e.g., Huawei P30 Pro 6, Xiaomi Mi 9), Windows Phone, and iOS (all Apple models).
- Chipset. Manages the data flow between the processor, memory, and peripherals.
- Apps. Application services to consumers.
- Durability.
- User-friendliness.
- Brand. A brand is associated with a company’s name and its services that distinguish it from other companies. The market analysis results show that ease of use of the device, access to spare parts and customer support are essential in attracting the customer to the product. High-quality service and the level of customer support can increase a company’s profit exponentially.
- Basic and extended in-built function. Basic built-in features are indispensable in all mobile phones. However, they are specific to particular phone models. For example, it is already a standard that every smartphone is equipped with a camera and a video camcorder. In addition, a smartphone also comes with GPS, a voice recorder, and a media player for video and audio files.
- Prominence, product reputation.
- Quality of the screen. There are smartphones on the market that are almost all screen, such as the Apple iPhone Xs. Super Retina HD OLED screens allow seeing color depth, such as the Apple iPhone Xs. Widespread and trusted by customers are SUPER AMOLED screens (an improved version of OLED screens). They are distinguished by energy efficiency and good contrast. In addition, they are ultra-thin and respond much faster to touch. High-end smartphones are often equipped with a protective layer of glass to protect against mechanical damage.

#### 1.2. Aim of the Study

## 2. Literature Review

#### 2.1. Characteristics and Applicability of Decision Support System

#### 2.2. Overview of Various MCDM Methods Applied to Decision-Making Problems

## 3. Materials and Methods

#### 3.1. The TOPSIS-COMET Method

**Step 1.**The dimensionality of the problem is defined by the expert by choosing the number r of criteria, ${C}_{1},{C}_{2},\dots ,{C}_{r}$. In the next stage, the set of fuzzy numbers for each criterion ${C}_{i}$ is chosen, i.e., ${\tilde{C}}_{i1},{\tilde{C}}_{i2},\dots ,{\tilde{C}}_{i{c}_{i}}$. Each fuzzy number defines the value of the membership for a particular linguistic concept for particular crisp values [91,92]. Therefore, it is also useful for variables that are not continuous. In this way, the following outcome is achieved (1).

**Step 2.**Characteristic objects are named generated objects that define reference points in n-dimensional space. Their role may be played by real or idealized objects, which do not exist [93]. The characteristic objects ($CO$) are received from the Cartesian product of fuzzy numbers cores for each criteria [43]. As the result, the ordered set of all $CO$ is obtained (2):

**Step 3.**The Matrix of Expert Judgement ($MEJ$) is determined by the expert. It is a result of pairwise comparison of the characteristic objects to the knowledge of the expert [94]. The $MEJ$ structure is represented by (4):

**Step 4.**Each characteristic object is converted into a fuzzy rule, where the degree of membership of each criterion is the premise for activating the inference in the form ${P}_{i}$. Each characteristic object and preference value is converted into a fuzzy rule in the following detailed form (7). In this way, a complete fuzzy rule base is obtained, which approximates the mental evaluation function of the expert ${f}_{exp}\left(C{O}_{i}\right)$ [95].

**Step 5.**Each one of the alternatives ${A}_{i}$ is a set of crisp numbers ${a}_{ri}$ corresponding to criteria ${C}_{1},{C}_{2},\dots ,{C}_{r}$. It can be displayed as follows (8):

#### 3.2. The COCOSO Method

**Step 1.**Definition of a decision matrix of dimension $n\times m$, where n is the number of alternatives, and m is the number of criteria (9).

**Step 2.**Normalization the decision matrix, where, for profit criteria, use the Equation (10), and for cost criteria, use the Equation (11). In this study, The Minimum-Maximum normalization method was used.

**Step 3.**Calculation of the weighted sum of the comparison sequence and the total power weight of the comparison sequences for each alternative. The values of ${S}_{i}$ are based on the grey relationship generation method (12), and for ${P}_{i}$ the values are achieved according to the multiplicative WASPAS setting (13).

**Step 4.**Computation of the relative weights of alternatives using aggregation strategies. The formulas determine the strategies (14)–(16), where the first strategy expresses the average of the sums of WSM and WPM scores (14), the second strategy expresses the sum of WSM and WPM scores over the best (15), and the third strategy expresses the compromise strategy of WSM and WPM by using the $\lambda $ value (16). In this study, a $\lambda $ value of 0.5 was used.

**Step 5.**Establish the final ranking of alternatives based on ${k}_{i}$ values defined using Formula (17). The higher the ${k}_{i}$ value, the higher the ranking.

#### 3.3. The EDAS Method

**Step 1.**Define a decision matrix of dimension $n\times m$, where n is the number of alternatives, and m is the number of criteria (18).

**Step 2.**Calculate the average solution for each criterion according to Formula (19).

**Step 3.**Calculating the positive distance from the mean solution and the negative distance from the mean solution for the alternatives. When the criterion is of profit type, the negative distance and the positive distance are calculated using Equations (21) and (20), while, when the criterion is of cost type, the distances are calculated using Formulas (23) and (22).

**Step 4.**Calculate the weighted sums of PDA and NDA for each decision variant using Equations (24) and (25).

**Step 5.**Normalize the weighted sums of negative and positive distances using Equations (26) and (27).

**Step 6.**Calculate the evaluation score ($AS$) for each alternative using Formula (28). A higher point value determines a higher ranking alternative.

#### 3.4. The MAIRCA Method

**Step 1.**Define a decision matrix of dimension $n\times m$, where n is the number of alternatives, and m is the number of criteria (29).

**Step 2.**Determine the preference for choosing alternatives using the vector ${P}_{Ai}$ using Formula (30).

**Step 3.**Create a theoretical ranking matrix ${T}_{p}$. The elements of this matrix are the multiplied priorities of alternatives by the criteria weights. The form of this matrix can be represented by Formula (32).

**Step 4.**Create the real rating matrix, which is shown by Formula (34).

**Step 5.**Calculate the total gap matrix (G) by taking the difference between the theoretical grade matrix ($Tp$) and the actual grade matrix ($Tr$) using Formula (37).

**Step 6.**Calculate the final values of the criterion functions (${Q}_{i}$) for the alternatives using the sum of the rows of the gap matrix (G) using Formula (38). The alternative with the lowest value of ${Q}_{i}$ has the highest ranking.

#### 3.5. The MABAC Method

**Step 1.**Define a decision matrix of dimension $n\times m$, where n is the number of alternatives, and m is the number of criteria (39).

**Step 2.**Normalization of the decision matrix, where, for criteria of type profit, Equation (40) is used, and, for criteria of type cost, Equation (41) is used.

**Step 3.**Create a weighted matrix based on the values from the normalized matrix according to Formula (42).

**Step 4.**Boundary approximation area (G) matrix determination. The Boundary Approximation Area ($BAA$) for all criteria can be determined using Formula (43).

**Step 5.**Distance calculation of alternatives from the boundary approximation area for matrix elements (Q) by Equation (44).

**Step 6.**Rank the alternatives according to the sum of the distances of the alternatives from the areas of approximation of the borders (46).

#### 3.6. The CRITIC Weighting Method

**Step 1.**Normalization of the initial decision matrix using Equation (47) for benefit criteria:

**Step 2.**Calculation of the correlation among criteria pairs by using Equation (48):

#### 3.7. Ranking Similarity Coefficients

#### Weighted Spearman’s Rank Correlation Coefficient

#### 3.8. Rank Similarity Coefficient

#### 3.9. The Copeland Method

**Step 1.**Evaluation of the alternatives. This technique first assigns $(n-1)$ points to the alternative which is most preferred by the decision-maker and zero points for the least preferred. This stage is performed for each MCDM method.

**Step 2.**Calculation of the Borda score. The Borda score of each alternative is received by summing up the scores.

**Step 3.**Rank the options. The best alternative has the highest Borda score.

**Step 1.**Calculation of the wins score. First of all, the wins score of each alternative is calculated. The win score is calculated as the sum of the ranking order of the alternatives according to different methods.

**Step 2.**Calculation of the losses score. The wins score of each option are subtracted from the majority wins score. In this way, the losses scores of the options are received.

**Step 3.**Determination of the final scores and ranking. The difference between the wins score and the losses score provides the final score of each option. Finally, the best option is this one, which has the highest overall score.

## 4. An Illustrative Case Study of a Multi-Criteria Mobile Phones Selection Problem with Proposed DSS

- In the first step, data from various online platforms are analyzed to recognize a list of relevant criteria used to assess mobile phones.
- The second step includes the calculation of the weights, which represent user preferences. In this illustrative example, the weights are determined using an objective weighting method named CRITIC, introduced in Section 3.6. However, it is also possible to customize to the decision-maker’s preference and use a vector of weights determined independently, or using another method, also subjective, such as AHP.
- Finally, a set of alternatives is evaluated using the MCDM methods, and a recommendation of the most advantageous product in the form of compromise ranking based on the used set of MCDMs is provided to a user.

## 5. Results

#### 5.1. A Small Set Containing 20 Sample Randomly Selected Alternatives from the Complete Set

#### Compromise Rankings Candidates and Their Sensitivity Analysis for the Little Set

#### 5.2. A Complete Set of 1039 Mobile Phones

#### Compromise Rankings Candidates and Their Sensitivity Analysis for the Whole Set

## 6. Discussion

## 7. Conclusions and Future Research Directions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Abbreviations

AHP | Analytic Hierarchy Process |

ANN | Artificial Neural Network |

ARAS | Additive Ratio Assessment |

BWM | Best-Worst Method |

COCOSO | COmbined COmpromise SOlution |

CODAS | COmbinative Distance-based ASsessment |

COMET | Characteristic Objects METhod |

CPU | Central Processing Unit |

CRITIC | Criteria Importance Through Inter-criteria Correlation |

DNN | Deep Neural Networks |

DSS | Decision Support Systems |

EDAS | Evaluation based on Distance from Average Solution |

ELECTRE | ÉLimination et Choix Traduisant la REalité |

EWP | Exponentially Weighted Product |

MABAC | Multiattribute Boundary Approximation Area Comparison |

MARICA | Multi-Attributive Real- Ideal Comparative Analysis |

MAUT | MultiAttribute Utility Theory |

MCDM | Multiple Criteria Decision-Making |

ML | Machine Learning |

MULTIMOORA | full Multiplicative form Multiple Objective Optimization |

on the basis of Ratio Analysis | |

OR-IDSS | Operations Research-based Intelligent Decision Support System |

OS | Operating System |

PROMETHEE | Preference Ranking Organization Method for Enrichment Evaluations |

RAM | Random Access Memory |

RS | Recommender Systems |

SAW | Simple Additive Weighting |

TOPSIS | Technique for Order of Preference by Similarity to Ideal Solution |

VIKOR | Vise Kriterijumska Optimizacija I Kompromisno Resenje |

WASPAS | step wise Weight assessment ratio analysis |

WS coefficient | Weighted Similarity coefficient |

## References

- Efe, B.; Yerlikaya, M.A.; Efe, Ö.F. Mobile phone selection based on a novel quality function deployment approach. Soft Comput.
**2020**, 24, 15447–15461. [Google Scholar] [CrossRef] - Lei, N.; Moon, S.K. A Decision Support System for market-driven product positioning and design. Decis. Support Syst.
**2015**, 69, 82–91. [Google Scholar] [CrossRef] - Deb, K.; Chatterjee, R.P.; Banerjee, S.; Bag, R.; Das, A. Mobile Phone Ranking by Analytical Hierarchical Process: A Case Study. Int. J. Inf. Eng. Electron. Bus.
**2018**, 11, 52. [Google Scholar] [CrossRef] - Wątróbski, J.; Jankowski, J.; Ziemba, P.; Karczmarczyk, A.; Zioło, M. Generalised framework for multi-criteria method selection. Omega
**2019**, 86, 107–124. [Google Scholar] [CrossRef] - Goswami, S.; Mitra, S. Selecting the best mobile model by applying AHP-COPRAS and AHP-ARAS decision making methodology. Int. J. Data Netw. Sci.
**2020**, 4, 27–42. [Google Scholar] [CrossRef] - Chen, I.F.; Tsaur, R.C.; Chen, P.Y. Selection of Best Smartphone using revised ELECTRE-III method. Int. J. Inf. Technol. Decis. Mak.
**2018**, 17, 1915–1936. [Google Scholar] [CrossRef] - Singh, R.; Avikal, S.; Rashmi, R.; Ram, M. A Kano model, AHP and TOPSIS based approach for selecting the best mobile phone under a fuzzy environment. Int. J. Qual. Reliab. Manag.
**2020**, 37, 837–851. [Google Scholar] [CrossRef] - Kumar, G.; Parimala, N. An integration of sentiment analysis and MCDM approach for smartphone recommendation. Int. J. Inf. Technol. Decis. Mak.
**2020**, 19, 1037–1063. [Google Scholar] [CrossRef] - Natasya, W.A.G.; Kusnawi, K. Decision support system design to decide on the latest smartphone using analytical hierarchy process. In Proceedings of the 2017 2nd International conferences on Information Technology, Information Systems and Electrical Engineering (ICITISEE), Yogyakarta, Indonesia, 1–2 November 2017; pp. 456–461. [Google Scholar]
- Aggarwal, A.; Choudhary, C.; Mehrotra, D. Evaluation of smartphones in Indian market using EDAS. Procedia Comput. Sci.
**2018**, 132, 236–243. [Google Scholar] [CrossRef] - Kumar, S.; Kumar, T.A.; Agrawal, S.; Singh, S.K. Research Methodology: Prioritization of New Smartphones Using TOPSIS and MOORA. In Proceedings of the International Conference of Advance Research & Innovation (ICARI), Delhi, India, 19 January 2020. [Google Scholar]
- Mishra, A.R.; Garg, A.K.; Purwar, H.; Rana, P.; Liao, H.; Mardani, A. An extended intuitionistic fuzzy multi-attributive border approximation area comparison approach for smartphone selection using discrimination measures. Informatica
**2021**, 32, 119–143. [Google Scholar] [CrossRef] - Musrini, M.; Umaroh, S.; Al Arifin, A. Desicion support system using topsis method for smartphone selection. In Proceedings of the 2nd Faculty of Industrial Technology International Congress International Conference (FoITIC 2020), Bandung, Indonesia, 28–30 January 2020; pp. 270–276. [Google Scholar]
- Okfalisa, O.; Rusnedy, H.; Iswavigra, D.U.; Pranggono, B.; Haerani, E.; Saktioto, T. Decision support system for smartphone recommendation: The comparison of fuzzy AHP and fuzzy ANP in multi-attribute decision making. SINERGI
**2021**, 25, 101–110. [Google Scholar] [CrossRef] - Sutton, R.T.; Pincock, D.; Baumgart, D.C.; Sadowski, D.C.; Fedorak, R.N.; Kroeker, K.I. An overview of clinical decision support systems: Benefits, risks, and strategies for success. NPJ Digit. Med.
**2020**, 3, 1–10. [Google Scholar] [CrossRef] [Green Version] - Acosta, M.; Corral, S. Multicriteria decision analysis and participatory decision support systems in forest management. Forests
**2017**, 8, 116. [Google Scholar] [CrossRef] - Alyoubi, B.A. Decision support system and knowledge-based strategic management. Procedia Comput. Sci.
**2015**, 65, 278–284. [Google Scholar] [CrossRef] [Green Version] - Vertakova, Y.; Mkrtchyan, V.; Leontyev, E. Information provision of decision support systems in conditions of structural changes and digitalization of the economy. J. Appl. Eng. Sci.
**2019**, 17, 74–80. [Google Scholar] [CrossRef] [Green Version] - Cánovas-Segura, B.; Campos, M.; Morales, A.; Juarez, J.M.; Palacios, F. Development of a clinical decision support system for antibiotic management in a hospital environment. Prog. Artif. Intell.
**2016**, 5, 181–197. [Google Scholar] [CrossRef] - Lahav, O.; Mastronarde, N.; van der Schaar, M. What is interpretable? using machine learning to design interpretable decision-support systems. arXiv
**2018**, arXiv:1811.10799. [Google Scholar] - Shoja, B.M.; Tabrizi, N. Customer reviews analysis with deep neural networks for e-commerce recommender systems. IEEE Access
**2019**, 7, 119121–119130. [Google Scholar] [CrossRef] - Baryannis, G.; Dani, S.; Validi, S.; Antoniou, G. Decision support systems and artificial intelligence in supply chain risk management. In Revisiting Supply Chain Risk; Springer: Berlin/Heidelberg, Germany, 2019; pp. 53–71. [Google Scholar]
- Jahani, A.; Feghhi, J.; Makhdoum, M.F.; Omid, M. Optimized forest degradation model (OFDM): An environmental decision support system for environmental impact assessment using an artificial neural network. J. Environ. Plan. Manag.
**2016**, 59, 222–244. [Google Scholar] [CrossRef] - Zong, K.; Yuan, Y.; Montenegro-Marin, C.E.; Kadry, S.N. Or-based intelligent decision support system for e-commerce. J. Theor. Appl. Electron. Commer. Res.
**2021**, 16, 1150–1164. [Google Scholar] [CrossRef] - Goguen, J. LA Zadeh. Fuzzy sets. Information and control, vol. 8 (1965), pp. 338–353.-LA Zadeh. Similarity relations and fuzzy orderings. Information sciences, vol. 3 (1971), pp. 177–200. J. Symb. Log.
**1973**, 38, 656–657. [Google Scholar] [CrossRef] - Yadegaridehkordi, E.; Nilashi, M.; Shuib, L.; Asadi, S.; Ibrahim, O. Development of a SaaS adoption decision-making model using a new hybrid MCDM approach. Int. J. Inf. Technol. Decis. Mak.
**2019**, 18, 1845–1874. [Google Scholar] [CrossRef] - Zhang, Y.; Abbas, H.; Sun, Y. Smart e-commerce integration with recommender systems. Electron. Mark.
**2019**, 29, 219–220. [Google Scholar] [CrossRef] [Green Version] - Alamdari, P.M.; Navimipour, N.J.; Hosseinzadeh, M.; Safaei, A.A.; Darwesh, A. A systematic study on the recommender systems in the E-commerce. IEEE Access
**2020**, 8, 115694–115716. [Google Scholar] [CrossRef] - Razmak, J.; Aouni, B. Decision Support System and Multi-Criteria Decision Aid: A State of the Art and Perspectives. J. Multi-Criteria Decis. Anal.
**2015**, 22, 101–117. [Google Scholar] [CrossRef] - Bączkiewicz, A.; Watrobski, J.; Sałabun, W. Towards MCDA Based Decision Support System Addressing Sustainable Assessment. 2021. Available online: https://aisel.aisnet.org/isd2014/proceedings2021/sustainable/6/ (accessed on 1 September 2021).
- Kizielewicz, B.; Więckowski, J.; Shekhovtsov, A.; Wątróbski, J.; Depczyński, R.; Sałabun, W. Study Towards The Time-based Mcda Ranking Analysis—A Supplier Selection Case Study. In Facta Universitatis, Series: Mechanical Engineering; University of Niš: Niš, Serbia, 2021. [Google Scholar]
- Khademolqorani, S.; Hamadani, A.Z. An adjusted decision support system through data mining and multiple criteria decision making. Procedia-Soc. Behav. Sci.
**2013**, 73, 388–395. [Google Scholar] [CrossRef] [Green Version] - Stanujkic, D.; Zavadskas, E.K.; Ghorabaee, M.K.; Turskis, Z. An extension of the EDAS method based on the use of interval grey numbers. Stud. Inform. Control
**2017**, 26, 5–12. [Google Scholar] [CrossRef] - Yazdani, M.; Chatterjee, P.; Montero-Simo, M.J.; Araque-Padilla, R.A. An integrated multi-attribute model for evaluation of sustainable mobile phone. Sustainability
**2019**, 11, 3704. [Google Scholar] [CrossRef] [Green Version] - Behzadian, M.; Otaghsara, S.K.; Yazdani, M.; Ignatius, J. A state-of the-art survey of TOPSIS applications. Expert Syst. Appl.
**2012**, 39, 13051–13069. [Google Scholar] [CrossRef] - Kuo, T. A modified TOPSIS with a different ranking index. Eur. J. Oper. Res.
**2017**, 260, 152–160. [Google Scholar] [CrossRef] - Žižović, M.; Pamučar, D.; Albijanić, M.; Chatterjee, P.; Pribićević, I. Eliminating rank reversal problem using a new multi-attribute model—The RAFSI method. Mathematics
**2020**, 8, 1015. [Google Scholar] [CrossRef] - Dezert, J.; Tchamova, A.; Han, D.; Tacnet, J.M. The SPOTIS rank reversal free method for multi-criteria decision-making support. In Proceedings of the 2020 IEEE 23rd International Conference on Information Fusion (FUSION), Rustenburg, South Africa, 6–9 July 2020; pp. 1–8. [Google Scholar]
- Sałabun, W. The Characteristic Objects Method: A New Distance-based Approach to Multicriteria Decision-making Problems. J. Multi-Criteria Decis. Anal.
**2015**, 22, 37–50. [Google Scholar] [CrossRef] - Kizielewicz, B.; Dobryakova, L. MCDA based approach to sports players’ evaluation under incomplete knowledge. Procedia Comput. Sci.
**2020**, 176, 3524–3535. [Google Scholar] [CrossRef] - Kizielewicz, B.; Dobryakova, L. How to choose the optimal single-track vehicle to move in the city? Electric scooters study case. Procedia Comput. Sci.
**2020**, 176, 2243–2253. [Google Scholar] [CrossRef] - Sałabun, W.; Palczewski, K.; Wątróbski, J. Multicriteria approach to sustainable transport evaluation under incomplete knowledge: Electric bikes case study. Sustainability
**2019**, 11, 3314. [Google Scholar] [CrossRef] [Green Version] - Sałabun, W.; Karczmarczyk, A. Using the comet method in the sustainable city transport problem: An empirical study of the electric powered cars. Procedia Comput. Sci.
**2018**, 126, 2248–2260. [Google Scholar] [CrossRef] - Shekhovtsov, A.; Kołodziejczyk, J.; Sałabun, W. Fuzzy Model Identification Using Monolithic and Structured Approaches in Decision Problems with Partially Incomplete Data. Symmetry
**2020**, 12, 1541. [Google Scholar] [CrossRef] - Yazdani, M.; Zarate, P.; Zavadskas, E.K.; Turskis, Z. A Combined Compromise Solution (CoCoSo) method for multi-criteria decision-making problems. Manag. Decis.
**2019**, 57, 2501–2519. [Google Scholar] [CrossRef] - Khan, S.; Haleem, A. Investigation of circular economy practices in the context of emerging economies: A CoCoSo approach. Int. J. Sustain. Eng.
**2021**, 1–11. [Google Scholar] [CrossRef] - Ecer, F. A consolidated MCDM framework for performance assessment of battery electric vehicles based on ranking strategies. Renew. Sustain. Energy Rev.
**2021**, 143, 110916. [Google Scholar] [CrossRef] - Yazdani, M.; Chatterjee, P. Intelligent decision making tools in manufacturing technology selection. In Futuristic Composites; Springer: Berlin/Heidelberg, Germany, 2018; pp. 113–126. [Google Scholar]
- Wen, Z.; Liao, H.; Mardani, A.; Al-Barakati, A. A hesitant fuzzy linguistic combined compromise solution method for multiple criteria decision making. In Proceedings of the International Conference on Management Science and Engineering Management, Chisinau, Moldova, 30 July–2 August 2020; Springer: Berlin/Heidelberg, Germany, 2019; pp. 813–821. [Google Scholar]
- Zolfani, S.H.; Chatterjee, P.; Yazdani, M. A structured framework for sustainable supplier selection using a combined BWM-CoCoSo model. In Proceedings of the International Scientific Conference in Business, Management and Economics Engineering, Vilnius, Lithuania, 9–10 May 2019; Vilnius Gediminas Technical University: Vilnius, Lithuania, 2019. [Google Scholar] [CrossRef] [Green Version]
- Zolfani, S.H.; Ecer, F.; Pamučar, D.; Raslanas, S. Neighborhood selection for a newcomer via a novel BWM-based revised MAIRCA integrated model: A case from the Coquimbo-La Serena conurbation, Chile. Int. J. Strateg. Prop. Manag.
**2020**, 24, 102–118. [Google Scholar] [CrossRef] - Pamučar, D.; Vasin, L.; Lukovac, L. Selection of railway level crossings for investing in security equipment using hybrid DEMATEL-MAIRCA model. In Proceedings of the XVI International Scientific-Expert Conference on Railway, Railcon, Niš, Serbia, 9–10 October 2014; pp. 89–92. [Google Scholar]
- Aksoy, E. An Analysis on Turkey’s Merger and Acquisition Activities: MAIRCA Method. Gümüşhane Üniversitesi Sos. Bilim. Enstitüsü Elektron. Derg.
**2021**, 12, 1–11. [Google Scholar] - Sari, F.; Kandemir, İ.; Ceylan, D.A.; Gül, A. Using AHP and PROMETHEE multi-criteria decision making methods to define suitable apiary locations. J. Apic. Res.
**2020**, 59, 546–557. [Google Scholar] [CrossRef] - Yalcin, N.; Ünlü, U. A multi-criteria performance analysis of Initial Public Offering (IPO) firms using CRITIC and VIKOR methods. Technol. Econ. Dev. Econ.
**2018**, 24, 534–560. [Google Scholar] [CrossRef] - Diakoulaki, D.; Mavrotas, G.; Papayannakis, L. Determining objective weights in multiple criteria problems: The CRITIC method. Comput. Oper. Res.
**1995**, 22, 763–770. [Google Scholar] [CrossRef] - Tuş, A.; Adalı, E.A. The new combination with CRITIC and WASPAS methods for the time and attendance software selection problem. Opsearch
**2019**, 56, 528–538. [Google Scholar] [CrossRef] - Kazan, H.; Ozdemir, O. Financial performance assessment of large scale conglomerates via TOPSIS and CRITIC methods. Int. J. Manag. Sustain.
**2014**, 3, 203–224. [Google Scholar] - Madic, M.; Radovanović, M. Ranking of some most commonly used nontraditional machining processes using ROV and CRITIC methods. UPB Sci. Bull. Ser. D
**2015**, 77, 193–204. [Google Scholar] - Zandi, I.; Pahlavani, P.; Bigdeli, B. Combining CRITIC Objective Weighting Method with CODAS and VIKOR Methods for Selecting Prospect Hospital Sites (Case Study: District 5 of Tehran). Geogr. Urban Space Dev.
**2021**. [Google Scholar] [CrossRef] - Adalı, E.A.; Işık, A.T. CRITIC and MAUT methods for the contract manufacturer selection problem. Eur. J. Multidiscip. Stud.
**2017**, 2, 93–101. [Google Scholar] [CrossRef] [Green Version] - Keshavarz Ghorabaee, M.; Amiri, M.; Kazimieras Zavadskas, E.; Antuchevičienė, J. Assessment of third-party logistics providers using a CRITIC–WASPAS approach with interval type-2 fuzzy sets. Transport
**2017**, 32, 66–78. [Google Scholar] [CrossRef] [Green Version] - Yang, M.; Zhou, X.; Xie, R.F.; GU, J.H.; Gu, X.J. Application of Analytic Hierarchy Process (AHP) Combined with CRITIC to Studying Weighted Coefficient of Multiple Criteria for Optimization of Extraction Technology of Compound Zishenqing Granules. Pharm. Care Res.
**2009**, 1. Available online: https://en.cnki.com.cn/Article_en/CJFDTotal-YXFY200901012.htm (accessed on 15 July 2021). - Mishra, A.R.; Rani, P.; Pandey, K. Fermatean fuzzy CRITIC-EDAS approach for the selection of sustainable third-party reverse logistics providers using improved generalized score function. J. Ambient. Intell. Humaniz. Comput.
**2021**, 1–17. [Google Scholar] [CrossRef] - Peng, X.; Zhang, X.; Luo, Z. Pythagorean fuzzy MCDM method based on CoCoSo and CRITIC with score function for 5G industry evaluation. Artif. Intell. Rev.
**2019**, 1–35. [Google Scholar] [CrossRef] - Peng, X.; Huang, H. Fuzzy decision making method based on CoCoSo with CRITIC for financial risk evaluation. Technol. Econ. Dev. Econ.
**2020**, 26, 695–724. [Google Scholar] [CrossRef] [Green Version] - Baidya, J.; Garg, H.; Saha, A.; Mishra, A.R.; Rani, P.; Dutta, D. Selection of third party reverses logistic providers: An approach of BCF-CRITIC-MULTIMOORA using Archimedean power aggregation operators. Complex Intell. Syst.
**2021**, 1–28. [Google Scholar] [CrossRef] - Goswami, S.S.; Behera, D.K. An Analysis for Selecting Best Smartphone Model by AHP-TOPSIS Decision-Making Methodology. Int. J. Serv. Sci. Manag. Eng. Technol. (IJSSMET)
**2021**, 12, 116–137. [Google Scholar] - Rani, P.; Mishra, A.R.; Ansari, M.D. Analysis of smartphone selection problem under interval-valued intuitionistic fuzzy ARAS and TOPSIS methods. In Proceedings of the 2019 Fifth International Conference on Image Information Processing (ICIIP), Shimla, India, 15–17 November 2019; pp. 509–514. [Google Scholar]
- Ansari, N. Smartphone Selection Using Analytic Hierarchy Process. 2016. Available online: http://dspace.sfit.co.in:8004/jspui/handle/123456789/1449 (accessed on 15 July 2021).
- Goswami, S.S.; Behera, D.K. Evaluation of the best smartphone model in the market by integrating fuzzy-AHP and PROMETHEE decision-making approach. Decision
**2021**, 48, 71–96. [Google Scholar] [CrossRef] - Mishra, A.R.; Singh, R.K.; Motwani, D. Intuitionistic fuzzy divergence measure-based ELECTRE method for performance of cellular mobile telephone service providers. Neural Comput. Appl.
**2020**, 32, 3901–3921. [Google Scholar] [CrossRef] - Mishra, A.R.; Rani, P. Shapley divergence measures with VIKOR method for multi-attribute decision-making problems. Neural Comput. Appl.
**2019**, 31, 1299–1316. [Google Scholar] [CrossRef] - Mishra, A.R.; Singh, R.K.; Motwani, D. Multi-criteria assessment of cellular mobile telephone service providers using intuitionistic fuzzy WASPAS method with similarity measures. Granul. Comput.
**2019**, 4, 511–529. [Google Scholar] [CrossRef] - Shekhovtsov, A.; Więckowski, J.; Kizielewicz, B.; Sałabun, W. Towards Reliable Decision-Making in the Green Urban Transport Domain; University of Niš: Niš, Serbia, 2020. [Google Scholar] [CrossRef]
- Sałabun, W.; Ziemba, P.; Wątróbski, J. The rank reversals paradox in management decisions: The comparison of the ahp and comet methods. In International Conference on Intelligent Decision Technologies; Springer: Berlin/Heidelberg, Germany, 2016; pp. 181–191. [Google Scholar]
- Faizi, S.; Rashid, T.; Sałabun, W.; Zafar, S.; Wątróbski, J. Decision making with uncertainty using hesitant fuzzy sets. Int. J. Fuzzy Syst.
**2018**, 20, 93–103. [Google Scholar] [CrossRef] [Green Version] - Sałabun, W.; Wątróbski, J.; Shekhovtsov, A. Are MCDA methods benchmarkable? A comparative study of TOPSIS, VIKOR, COPRAS, and PROMETHEE II methods. Symmetry
**2020**, 12, 1549. [Google Scholar] [CrossRef] - García-Cascales, M.S.; Lamata, M.T. On rank reversal and TOPSIS method. Math. Comput. Model.
**2012**, 56, 123–132. [Google Scholar] [CrossRef] - Yang, W. Ingenious solution for the rank reversal problem of TOPSIS method. Math. Probl. Eng.
**2020**, 2020, 9676518. [Google Scholar] [CrossRef] [Green Version] - Barzilai, J.; Golany, B. AHP rank reversal, normalization and aggregation rules. INFOR Inf. Syst. Oper. Res.
**1994**, 32, 57–64. [Google Scholar] [CrossRef] - Verly, C.; De Smet, Y. Some results about rank reversal instances in the PROMETHEE methods. Int. J. Multicriteria Decis. Mak. 71
**2013**, 3, 325–345. [Google Scholar] [CrossRef] - Liu, X.; Ma, Y. A method to analyze the rank reversal problem in the ELECTRE II method. Omega
**2021**, 102, 102317. [Google Scholar] [CrossRef] - Sałabun, W.; Piegat, A.; Wątróbski, J.; Karczmarczyk, A.; Jankowski, J. The COMET Method: The First MCDA Method Completely Resistant to Rank Reversal Paradox; European Working Group “Multiple Criteria Decision Aiding” Printemps 2019, Series 3, No. 39; EWG-MCDA: Cracow, Poland, Spring; 2019; Available online: http://www.cs.put.poznan.pl/ewgmcda/newsletter/Spr19.pdf (accessed on 15 July 2021).
- Piegat, A.; Sałabun, W. Identification of a multicriteria decision-making model using the characteristic objects method. Appl. Comput. Intell. Soft Comput.
**2014**, 2014, 536492. [Google Scholar] [CrossRef] [Green Version] - Sałabun, W. Fuzzy Multi-Criteria Decision-Making Method: The Modular Approach in the Characteristic Objects Method. Stud. Proc. Pol. Assoc. Knowl. Manag.
**2015**, 77, 54–64. [Google Scholar] - Kizielewicz, B.; Shekhovtsov, A.; Sałabun, W. A New Approach to Eliminate Rank Reversal in the MCDA Problems. In Proceedings of the International Conference on Computational Science, Cracow, Poland, 16–18 June 2021; Springer: Berlin/Heidelberg, Germany, 2021; pp. 338–351. [Google Scholar]
- Jiří, M. The robustness of TOPSIS results using sensitivity analysis based on weight tuning. In World Congress on Medical Physics and Biomedical Engineering 2018; Springer: Singapore, 2019; pp. 83–86. [Google Scholar]
- Shekhovtsov, A.; Kizielewicz, B.; Sałabun, W. New Rank-Reversal Free Approach to Handle Interval Data in MCDA Problems. In Proceedings of the International Conference on Computational Science, Cracow, Poland, 16–18 June 2021; Springer: Berlin/Heidelberg, Germany, 2021; pp. 458–472. [Google Scholar]
- Sałabun, W.; Karczmarczyk, A.; Wątróbski, J.; Jankowski, J. Handling data uncertainty in decision making with COMET. In Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence (SSCI), Bangalore, India, 18–21 November 2018; pp. 1478–1484. [Google Scholar]
- Faizi, S.; Sałabun, W.; Rashid, T.; Wątróbski, J.; Zafar, S. Group decision-making for hesitant fuzzy sets based on characteristic objects method. Symmetry
**2017**, 9, 136. [Google Scholar] [CrossRef] - Sałabun, W.; Karczmarczyk, A.; Wątróbski, J. Decision-making using the hesitant fuzzy sets COMET method: An empirical study of the electric city buses selection. In Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence (SSCI), Bangalore, India, 18–21 November 2018; pp. 1485–1492. [Google Scholar]
- Sałabun, W. Reduction in the number of comparisons required to create matrix of expert judgment in the comet method. Manag. Prod. Eng. Rev.
**2014**, 5, 62–69. [Google Scholar] [CrossRef] - Kizielewicz, B.; Kołodziejczyk, J. Effects of the selection of characteristic values on the accuracy of results in the COMET method. Procedia Comput. Sci.
**2020**, 176, 3581–3590. [Google Scholar] [CrossRef] - Wątróbski, J.; Sałabun, W.; Karczmarczyk, A.; Wolski, W. Sustainable decision-making using the COMET method: An empirical study of the ammonium nitrate transport management. In Proceedings of the 2017 Federated Conference on Computer Science and Information Systems (FedCSIS), Prague, Czech Republic, 3–6 September 2017; pp. 949–958. [Google Scholar]
- Lai, H.; Liao, H.; Wen, Z.; Zavadskas, E.K.; Al-Barakati, A. An Improved CoCoSo Method with a Maximum Variance Optimization Model for Cloud Service Provider Selection. Eng. Econ.
**2020**, 31, 411–424. [Google Scholar] [CrossRef] - Afshari, A.; Mojahed, M.; Yusuff, R.M. Simple additive weighting approach to personnel selection problem. Int. J. Innov. Manag. Technol.
**2010**, 1, 511. [Google Scholar] - Zavadskas, E.K.; Turskis, Z.; Antucheviciene, J.; Zakarevicius, A. Optimization of weighted aggregated sum product assessment. Elektron. Ir Elektrotechnika
**2012**, 122, 3–6. [Google Scholar] [CrossRef] - Zanakis, S.H.; Solomon, A.; Wishart, N.; Dublish, S. Multi-attribute decision making: A simulation comparison of select methods. Eur. J. Oper. Res.
**1998**, 107, 507–529. [Google Scholar] [CrossRef] - Wei, J.; Lin, X. The multiple attribute decision-making VIKOR method and its application. In Proceedings of the 2008 4th international conference on wireless communications, networking and mobile computing, Dalian, China, 12–14 October 2008; pp. 1–4. [Google Scholar]
- Brauers, W.K.; Zavadskas, E.K. The MOORA method and its application to privatization in a transition economy. Control Cybern.
**2006**, 35, 445–469. [Google Scholar] - Yazdani, M.; Wen, Z.; Liao, H.; Banaitis, A.; Turskis, Z. A grey combined compromise solution (CoCoSo-G) method for supplier selection in construction management. J. Civ. Eng. Manag.
**2019**, 25, 858–874. [Google Scholar] [CrossRef] [Green Version] - Keshavarz Ghorabaee, M.; Zavadskas, E.K.; Olfat, L.; Turskis, Z. Multi-criteria inventory classification using a new method of evaluation based on distance from average solution (EDAS). Informatica
**2015**, 26, 435–451. [Google Scholar] [CrossRef] - Ghorabaee, M.K.; Zavadskas, E.K.; Amiri, M.; Turskis, Z. Extended EDAS method for fuzzy multi-criteria decision-making: An application to supplier selection. Int. J. Comput. Commun. Control
**2016**, 11, 358–371. [Google Scholar] [CrossRef] [Green Version] - Pamučar, D.; Mihajlović, M.; Obradović, R.; Atanasković, P. Novel approach to group multi-criteria decision making based on interval rough numbers: Hybrid DEMATEL-ANP-MAIRCA model. Expert Syst. Appl.
**2017**, 88, 58–80. [Google Scholar] [CrossRef] [Green Version] - Gigović, L.; Pamučar, D.; Bajić, Z.; Milićević, M. The combination of expert judgment and GIS-MAIRCA analysis for the selection of sites for ammunition depots. Sustainability
**2016**, 8, 372. [Google Scholar] [CrossRef] [Green Version] - Pamučar, D.; Ćirović, G. The selection of transport and handling resources in logistics centers using Multi-Attributive Border Approximation area Comparison (MABAC). Expert Syst. Appl.
**2015**, 42, 3016–3028. [Google Scholar] [CrossRef] - Wei, G.; Wei, C.; Wu, J.; Wang, H. Supplier selection of medical consumption products with a probabilistic linguistic MABAC method. Int. J. Environ. Res. Public Health
**2019**, 16, 5082. [Google Scholar] [CrossRef] [Green Version] - Jahan, A.; Mustapha, F.; Sapuan, S.; Ismail, M.Y.; Bahraminasab, M. A framework for weighting of criteria in ranking stage of material selection process. Int. J. Adv. Manuf. Technol.
**2012**, 58, 411–420. [Google Scholar] [CrossRef] - Sałabun, W.; Urbaniak, K. A new coefficient of rankings similarity in decision-making problems. In Proceedings of the International Conference on Computational Science, Amsterdam, The Netherlands, 3–5 June 2020; Springer: Berlin/Heidelberg, Germany, 2020; pp. 632–645. [Google Scholar]
- Shekhovtsov, A.; Sałabun, W. A comparative case study of the VIKOR and TOPSIS rankings similarity. Procedia Comput. Sci.
**2020**, 176, 3730–3740. [Google Scholar] [CrossRef] - Shekhovtsov, A.; Kozlov, V.; Nosov, V.; Sałabun, W. Efficiency of Methods for Determining the Relevance of Criteria in Sustainable Transport Problems: A Comparative Case Study. Sustainability
**2020**, 12, 7915. [Google Scholar] [CrossRef]

**Figure 2.**Visualization of rankings correlation coefficients ${r}_{w}$ and $WS$ for each pair of MCDM method for a small set of alternatives.

**Figure 3.**Visualization of rankings correlation coefficients ${r}_{w}$ and $WS$ for each pair of MCDM method for the whole set of alternatives.

MCDM Method Acronym | Year | References |
---|---|---|

AHP | 2016 | [70] |

2018 | [3] | |

AHP, BWM and DEMATEL | 2019 | [34] |

AHP-COPRAS, AHP-ARAS | 2020 | [5] |

Fuzzy AHP, Fuzzy ANP | 2021 | [14] |

Fuzzy AHP and PROMETHEE II | 2021 | [71] |

Fuzzy ARAS and TOPSIS | 2019 | [69] |

COCOSO | 2019 | [34] |

EDAS | 2018 | [10] |

ELECTRE | 2020 | [72] |

ELECTRE-III | 2018 | [6] |

MABAC | 2021 | [12] |

TOPSIS | 2020 | [13] |

TOPSIS combined with AHP as weighting method | 2020 | [7] |

2020 | [8] | |

2021 | [68] | |

TOPSIS and MOORA | 2020 | [11] |

VIKOR | 2019 | [73] |

WASPAS | 2019 | [74] |

${\mathit{C}}_{\mathit{i}}$ | Name | Type | Unit |
---|---|---|---|

${C}_{1}$ | Weight | Cost | Gram $\left[g\right]$ |

${C}_{2}$ | Weight | Cost | Ounce $\left[oz\right]$ |

${C}_{3}$ | Year of production | Profit | Year |

${C}_{4}$ | Display resolution | Profit | Megapixel $\left[Mpix\right]$ |

${C}_{5}$ | Memory card | Profit | Gigabyte $\left[GB\right]$ |

${C}_{6}$ | Internal memory | Profit | Gigabyte $\left[GB\right]$ |

${C}_{7}$ | Random-access memory (RAM) | Profit | Megabyte $\left[MB\right]$ |

${C}_{8}$ | Primary camera | Profit | Megapixel $\left[Mpix\right]$ |

${C}_{9}$ | Secondary camera | Profit | Megapixel $\left[Mpix\right]$ |

${C}_{10}$ | Bluetooth | Profit | Version |

${C}_{11}$ | USB | Profit | Version |

${C}_{12}$ | Approx price | Cost | Euro [€] |

${\mathit{C}}_{1}$ | ${\mathit{C}}_{2}$ | ${\mathit{C}}_{3}$ | ${\mathit{C}}_{4}$ | ${\mathit{C}}_{5}$ | ${\mathit{C}}_{6}$ | ${\mathit{C}}_{7}$ | ${\mathit{C}}_{8}$ | ${\mathit{C}}_{9}$ | ${\mathit{C}}_{10}$ | ${\mathit{C}}_{11}$ | ${\mathit{C}}_{12}$ |
---|---|---|---|---|---|---|---|---|---|---|---|

0.07703 | 0.09694 | 0.06047 | 0.07337 | 0.10404 | 0.05306 | 0.05944 | 0.09119 | 0.08999 | 0.07846 | 0.13067 | 0.08529 |

Phones | ${\mathit{C}}_{1}$ | ${\mathit{C}}_{2}$ | ${\mathit{C}}_{3}$ | ${\mathit{C}}_{4}$ | ${\mathit{C}}_{5}$ | ${\mathit{C}}_{6}$ | ${\mathit{C}}_{7}$ | ${\mathit{C}}_{8}$ | ${\mathit{C}}_{9}$ | ${\mathit{C}}_{10}$ | ${\mathit{C}}_{11}$ | ${\mathit{C}}_{12}$ |
---|---|---|---|---|---|---|---|---|---|---|---|---|

KP110 | 202 | 7 | 2015 | 5 | 64 | 8 | 2048 | 13 | 5 | 4 | 2 | 170 |

C3-01 Gold Edition | 150 | 5 | 2014 | 5 | 32 | 4 | 1024 | 8 | 2 | 4 | 2 | 180 |

7380 | 160 | 5 | 2016 | 5 | 256 | 32 | 3072 | 16 | 8 | 4 | 1 | 420 |

W180 | 170 | 6 | 2016 | 5 | 32 | 16 | 2048 | 8 | 8 | 4 | 2 | 110 |

Aquaris M4.5 | 115 | 6 | 2015 | 4 | 64 | 8 | 1024 | 8 | 5 | 4 | 2 | 180 |

Zenfone 3 Deluxe 5.5 ZS550KL | 151 | 5 | 2016 | 5 | 256 | 64 | 4096 | 16 | 8 | 4 | 2 | 440 |

M600 | 310 | 10 | 2017 | 8 | 256 | 16 | 2048 | 5 | 2 | 4 | 2 | 100 |

Liquid Z410 | 145 | 11 | 2015 | 4 | 32 | 8 | 1024 | 5 | 2 | 4 | 2 | 130 |

Liquid Jade Z | 110 | 3 | 2015 | 5 | 32 | 8 | 1024 | 13 | 5 | 4 | 2 | 200 |

nova plus | 160 | 5 | 2016 | 5 | 256 | 32 | 3072 | 16 | 8 | 4 | 1 | 420 |

X3-02 Touch and Type | 130 | 4 | 2013 | 4 | 64 | 8 | 1024 | 8 | 1 | 4 | 2 | 290 |

Zenfone 2 ZE551ML | 170 | 6 | 2015 | 5 | 256 | 16 | 2048 | 13 | 5 | 4 | 2 | 370 |

1110i | 143 | 4 | 2017 | 5 | 256 | 16 | 2048 | 13 | 5 | 4 | 2 | 150 |

P5 Qmax | 176 | 6 | 2013 | 5 | 32 | 4 | 1024 | 8 | 1 | 4 | 2 | 230 |

Fonepad 8 FE380CG | 328 | 11 | 2014 | 8 | 64 | 8 | 1024 | 5 | 2 | 4 | 2 | 200 |

XDA Atom Exec | 132 | 4 | 2012 | 4 | 32 | 8 | 1024 | 8 | 1 | 3 | 2 | 210 |

Idol 4 | 135 | 4 | 2016 | 5 | 256 | 16 | 2048 | 13 | 8 | 4 | 2 | 250 |

Rezound | 170 | 6 | 2011 | 4 | 32 | 16 | 1024 | 8 | 2 | 3 | 2 | 160 |

V295 | 150 | 5 | 2014 | 5 | 32 | 8 | 1024 | 13 | 5 | 4 | 2 | 190 |

Honor 6A | 143 | 4 | 2017 | 5 | 256 | 16 | 2048 | 13 | 5 | 4 | 2 | 150 |

${\mathit{A}}_{\mathit{i}}$ | ${\mathit{C}}_{1}$ | ${\mathit{C}}_{2}$ | ${\mathit{C}}_{3}$ | ${\mathit{C}}_{4}$ | ${\mathit{C}}_{5}$ | ${\mathit{C}}_{6}$ | ${\mathit{C}}_{7}$ | ${\mathit{C}}_{8}$ | ${\mathit{C}}_{9}$ | ${\mathit{C}}_{10}$ | ${\mathit{C}}_{11}$ | ${\mathit{C}}_{12}$ |
---|---|---|---|---|---|---|---|---|---|---|---|---|

${A}_{1}$ | 0.3841 | 0.3636 | 0.9990 | 0.6250 | 0.2500 | 0.1250 | 0.5000 | 0.8125 | 0.6250 | 1.0000 | 1.0000 | 0.6136 |

${A}_{2}$ | 0.5427 | 0.5455 | 0.9985 | 0.6250 | 0.1250 | 0.0625 | 0.2500 | 0.5000 | 0.2500 | 1.0000 | 1.0000 | 0.5909 |

${A}_{3}$ | 0.5122 | 0.5455 | 0.9995 | 0.6250 | 1.0000 | 0.5000 | 0.7500 | 1.0000 | 1.0000 | 1.0000 | 0.5000 | 0.0455 |

${A}_{4}$ | 0.4817 | 0.4545 | 0.9995 | 0.6250 | 0.1250 | 0.2500 | 0.5000 | 0.5000 | 1.0000 | 1.0000 | 1.0000 | 0.7500 |

${A}_{5}$ | 0.6494 | 0.4545 | 0.9990 | 0.5000 | 0.2500 | 0.1250 | 0.2500 | 0.5000 | 0.6250 | 1.0000 | 1.0000 | 0.5909 |

${A}_{6}$ | 0.5396 | 0.5455 | 0.9995 | 0.6250 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 0.0000 |

${A}_{7}$ | 0.0549 | 0.0909 | 1.0000 | 1.0000 | 1.0000 | 0.2500 | 0.5000 | 0.3125 | 0.2500 | 1.0000 | 1.0000 | 0.7727 |

${A}_{8}$ | 0.5579 | 0.0000 | 0.9990 | 0.5000 | 0.1250 | 0.1250 | 0.2500 | 0.3125 | 0.2500 | 1.0000 | 1.0000 | 0.7045 |

${A}_{9}$ | 0.6646 | 0.7273 | 0.9990 | 0.6250 | 0.1250 | 0.1250 | 0.2500 | 0.8125 | 0.6250 | 1.0000 | 1.0000 | 0.5455 |

${A}_{10}$ | 0.5122 | 0.5455 | 0.9995 | 0.6250 | 1.0000 | 0.5000 | 0.7500 | 1.0000 | 1.0000 | 1.0000 | 0.5000 | 0.0455 |

${A}_{11}$ | 0.6037 | 0.6364 | 0.9980 | 0.5000 | 0.2500 | 0.1250 | 0.2500 | 0.5000 | 0.1250 | 1.0000 | 1.0000 | 0.3409 |

${A}_{12}$ | 0.4817 | 0.4545 | 0.9990 | 0.6250 | 1.0000 | 0.2500 | 0.5000 | 0.8125 | 0.6250 | 1.0000 | 1.0000 | 0.1591 |

${A}_{13}$ | 0.5640 | 0.6364 | 1.0000 | 0.6250 | 1.0000 | 0.2500 | 0.5000 | 0.8125 | 0.6250 | 1.0000 | 1.0000 | 0.6591 |

${A}_{14}$ | 0.4634 | 0.4545 | 0.9980 | 0.6250 | 0.1250 | 0.0625 | 0.2500 | 0.5000 | 0.1250 | 1.0000 | 1.0000 | 0.4773 |

${A}_{15}$ | 0.0000 | 0.0000 | 0.9985 | 1.0000 | 0.2500 | 0.1250 | 0.2500 | 0.3125 | 0.2500 | 1.0000 | 1.0000 | 0.5455 |

${A}_{16}$ | 0.5976 | 0.6364 | 0.9975 | 0.5000 | 0.1250 | 0.1250 | 0.2500 | 0.5000 | 0.1250 | 0.7500 | 1.0000 | 0.5227 |

${A}_{17}$ | 0.5884 | 0.6364 | 0.9995 | 0.6250 | 1.0000 | 0.2500 | 0.5000 | 0.8125 | 1.0000 | 1.0000 | 1.0000 | 0.4318 |

${A}_{18}$ | 0.4817 | 0.4545 | 0.9970 | 0.5000 | 0.1250 | 0.2500 | 0.2500 | 0.5000 | 0.2500 | 0.7500 | 1.0000 | 0.6364 |

${A}_{19}$ | 0.5427 | 0.5455 | 0.9985 | 0.6250 | 0.1250 | 0.1250 | 0.2500 | 0.8125 | 0.6250 | 1.0000 | 1.0000 | 0.5682 |

${A}_{20}$ | 0.5640 | 0.6364 | 1.0000 | 0.6250 | 1.0000 | 0.2500 | 0.5000 | 0.8125 | 0.6250 | 1.0000 | 1.0000 | 0.6591 |

${\mathit{A}}_{\mathit{i}}$ | TOPSIS-COMET | COCOSO | EDAS | MAIRCA | MABAC |
---|---|---|---|---|---|

${A}_{1}$ | 10 | 6 | 10 | 11 | 11 |

${A}_{2}$ | 14 | 14 | 14 | 14 | 14 |

${A}_{3}$ | 11.5 | 7.5 | 5.5 | 6.5 | 6.5 |

${A}_{4}$ | 7 | 9 | 8 | 9 | 9 |

${A}_{5}$ | 13 | 13 | 13 | 13 | 13 |

${A}_{6}$ | 1 | 4 | 1 | 1 | 1 |

${A}_{7}$ | 8 | 10 | 12 | 10 | 10 |

${A}_{8}$ | 18 | 18 | 19 | 17 | 17 |

${A}_{9}$ | 6 | 11 | 9 | 8 | 8 |

${A}_{10}$ | 11 | 7.5 | 5.5 | 6.5 | 6.5 |

${A}_{11}$ | 15 | 15 | 16 | 15 | 15 |

${A}_{12}$ | 5 | 5 | 7 | 5 | 5 |

${A}_{13}$ | 3.5 | 2.5 | 3.5 | 3.5 | 3.5 |

${A}_{14}$ | 16 | 16 | 18 | 16 | 16 |

${A}_{15}$ | 19 | 17 | 20 | 18 | 18 |

${A}_{16}$ | 17 | 19 | 17 | 19 | 19 |

${A}_{17}$ | 2 | 1 | 2 | 2 | 2 |

${A}_{18}$ | 20 | 20 | 15 | 20 | 20 |

${A}_{19}$ | 9 | 12 | 11 | 12 | 12 |

${A}_{20}$ | 3.5 | 2.5 | 3.5 | 3.5 | 3.5 |

${\mathit{A}}_{\mathit{i}}$ | Copeland | Excluded Method Ranking | ||||
---|---|---|---|---|---|---|

Rank | $\mathbf{\text{TOPSIS-COMET}}$ | $\mathbf{COCOSO}$ | $\mathbf{EDAS}$ | $\mathbf{MAIRCA}$ | $\mathbf{MABAC}$ | |

${A}_{1}$ | 10 | 10 | 11 | 11 | 10 | 10 |

${A}_{2}$ | 14 | 14 | 14 | 14 | 14 | 14 |

${A}_{3}$ | 7 | 7 | 7 | 7 | 7 | 7 |

${A}_{4}$ | 9 | 8 | 9 | 9 | 8 | 8 |

${A}_{5}$ | 13 | 13 | 13 | 13 | 13 | 13 |

${A}_{6}$ | 1 | 2 | 1 | 2 | 2 | 2 |

${A}_{7}$ | 11 | 11 | 10 | 10 | 11 | 11 |

${A}_{8}$ | 17 | 17 | 17 | 17 | 17 | 17 |

${A}_{9}$ | 8 | 9 | 8 | 8 | 9 | 9 |

${A}_{10}$ | 6 | 6 | 6 | 6 | 6 | 6 |

${A}_{11}$ | 15 | 15 | 15 | 15 | 15 | 15 |

${A}_{12}$ | 5 | 5 | 5 | 5 | 5 | 5 |

${A}_{13}$ | 4 | 4 | 4 | 4 | 4 | 4 |

${A}_{14}$ | 16 | 16 | 16 | 16 | 16 | 16 |

${A}_{15}$ | 19 | 18 | 19 | 18 | 19 | 19 |

${A}_{16}$ | 18 | 19 | 18 | 19 | 18 | 18 |

${A}_{17}$ | 2 | 1 | 2 | 1 | 1 | 1 |

${A}_{18}$ | 20 | 20 | 20 | 20 | 20 | 20 |

${A}_{19}$ | 12 | 12 | 12 | 12 | 12 | 12 |

${A}_{20}$ | 3 | 3 | 3 | 3 | 3 | 3 |

**Table 8.**Values of similarity coefficient ${r}_{w}$ for compromise ranking candidates with consideration of sample smartphones.

${\mathit{r}}_{\mathit{w}}$ | Method Rank | Mean | Min | ||||
---|---|---|---|---|---|---|---|

$\mathbf{\text{TOPSIS-COMET}}$ | $\mathbf{COCOSO}$ | $\mathbf{EDAS}$ | $\mathbf{MAIRCA}$ | $\mathbf{MABAC}$ | |||

Rank 1 | 0.9334 | 0.9566 | 0.9803 | 0.9970 | 0.9970 | 0.9729 | 0.9334 |

Rank 2 | 0.9279 | 0.9689 | 0.9782 | 0.9928 | 0.9928 | 0.9721 | 0.9279 |

Rank 3 | 0.9366 | 0.9499 | 0.9773 | 0.9985 | 0.9985 | 0.9722 | 0.9366 |

Rank 4 | 0.9330 | 0.9588 | 0.9735 | 0.9961 | 0.9961 | 0.9715 | 0.9330 |

Rank 5 | 0.9287 | 0.9681 | 0.9793 | 0.9924 | 0.9924 | 0.9722 | 0.9287 |

Rank 6 | 0.9287 | 0.9681 | 0.9793 | 0.9924 | 0.9924 | 0.9722 | 0.9287 |

**Table 9.**Values of similarity coefficient $WS$ for compromise ranking candidates with consideration of sample smartphones.

WS | Method Rank | Mean | Min | ||||
---|---|---|---|---|---|---|---|

$\mathbf{\text{TOPSIS-COMET}}$ | $\mathbf{COCOSO}$ | $\mathbf{EDAS}$ | $\mathbf{MAIRCA}$ | $\mathbf{MABAC}$ | |||

Rank 1 | 0.9843 | 0.8942 | 0.9882 | 0.9934 | 0.9934 | 0.9707 | 0.8942 |

Rank 2 | 0.9443 | 0.9596 | 0.9485 | 0.9527 | 0.9527 | 0.9515 | 0.9443 |

Rank 3 | 0.9842 | 0.8944 | 0.9880 | 0.9935 | 0.9935 | 0.9707 | 0.8944 |

Rank 4 | 0.9440 | 0.9595 | 0.9478 | 0.9533 | 0.9533 | 0.9516 | 0.9440 |

Rank 5 | 0.9443 | 0.9596 | 0.9485 | 0.9527 | 0.9527 | 0.9515 | 0.9443 |

Rank 6 | 0.9443 | 0.9596 | 0.9485 | 0.9527 | 0.9527 | 0.9515 | 0.9443 |

**Table 10.**Correlation matrix for ${r}_{w}$ values for compromise ranking candidates with consideration of sample smartphones.

${\mathit{r}}_{\mathit{w}}$ | Rank 1 | Rank 2 | Rank 3 | Rank 4 | Rank 5 | Rank 6 |
---|---|---|---|---|---|---|

Rank 1 | 1.0000 | 0.9951 | 0.9985 | 0.9953 | 0.9954 | 0.9954 |

Rank 2 | 0.9951 | 1.0000 | 0.9936 | 0.9967 | 0.9996 | 0.9996 |

Rank 3 | 0.9985 | 0.9936 | 1.0000 | 0.9968 | 0.9939 | 0.9939 |

Rank 4 | 0.9953 | 0.9967 | 0.9968 | 1.0000 | 0.9963 | 0.9963 |

Rank 5 | 0.9954 | 0.9996 | 0.9939 | 0.9963 | 1.0000 | 1.0000 |

Rank 6 | 0.9954 | 0.9996 | 0.9939 | 0.9963 | 1.0000 | 1.0000 |

**Table 11.**Correlation matrix for $WS$ values for compromise ranking candidates with consideration of sample smartphones.

WS | Rank 1 | Rank 2 | Rank 3 | Rank 4 | Rank 5 | Rank 6 |
---|---|---|---|---|---|---|

Rank 1 | 1.0000 | 0.9593 | 0.9999 | 0.9596 | 0.9593 | 0.9593 |

Rank 2 | 0.9593 | 1.0000 | 0.9591 | 0.9994 | 1.0000 | 1.0000 |

Rank 3 | 0.9999 | 0.9591 | 1.0000 | 0.9598 | 0.9591 | 0.9591 |

Rank 4 | 0.9596 | 0.9994 | 0.9598 | 1.0000 | 0.9994 | 0.9994 |

Rank 5 | 0.9593 | 1.0000 | 0.9591 | 0.9994 | 1.0000 | 1.0000 |

Rank 6 | 0.9593 | 1.0000 | 0.9591 | 0.9994 | 1.0000 | 1.0000 |

${\mathit{C}}_{1}$ | ${\mathit{C}}_{2}$ | ${\mathit{C}}_{3}$ | ${\mathit{C}}_{4}$ | ${\mathit{C}}_{5}$ | ${\mathit{C}}_{6}$ | ${\mathit{C}}_{7}$ | ${\mathit{C}}_{8}$ | ${\mathit{C}}_{9}$ | ${\mathit{C}}_{10}$ | ${\mathit{C}}_{11}$ | ${\mathit{C}}_{12}$ |
---|---|---|---|---|---|---|---|---|---|---|---|

0.10300 | 0.11423 | 0.11538 | 0.10305 | 0.11780 | 0.06481 | 0.03916 | 0.10286 | 0.03233 | 0.10192 | 0.07646 | 0.02895 |

Phones | ${\mathit{C}}_{1}$ | ${\mathit{C}}_{2}$ | ${\mathit{C}}_{3}$ | ${\mathit{C}}_{4}$ | ${\mathit{C}}_{5}$ | ${\mathit{C}}_{6}$ | ${\mathit{C}}_{7}$ | ${\mathit{C}}_{8}$ | ${\mathit{C}}_{9}$ | ${\mathit{C}}_{10}$ | ${\mathit{C}}_{11}$ | ${\mathit{C}}_{12}$ |
---|---|---|---|---|---|---|---|---|---|---|---|---|

Iconia Talk S | 260 | 9 | 2016 | 7 | 128 | 16 | 2048 | 13 | 2 | 4 | 2 | 170 |

Liquid Z6 Plus | 169 | 5 | 2016 | 5 | 256 | 32 | 3072 | 13 | 5 | 4 | 2 | 250 |

Liquid X2 | 166 | 5 | 2015 | 5 | 32 | 32 | 3072 | 13 | 13 | 4 | 2 | 230 |

Liquid Zest | 125 | 4 | 2016 | 5 | 32 | 8 | 1024 | 8 | 5 | 4 | 2 | 110 |

Predator 8 | 353 | 12 | 2015 | 8 | 256 | 32 | 2048 | 5 | 2 | 4 | 2 | 350 |

Liquid Jade Primo | 150 | 5 | 2015 | 5 | 256 | 32 | 3072 | 21 | 8 | 4 | 3 | 220 |

Iconia Tab 10 A3-A30 | 540 | 1 | 2015 | 10 | 256 | 16 | 2048 | 5 | 2 | 4 | 2 | 250 |

Iconia Tab A3-A20 | 508 | 12 | 2014 | 10 | 256 | 16 | 1024 | 5 | 2 | 4 | 2 | 190 |

Iconia Tab A3-A20FHD | 508 | 12 | 2014 | 10 | 256 | 32 | 2048 | 5 | 2 | 4 | 2 | 230 |

Liquid Jade Z | 110 | 3 | 2015 | 5 | 32 | 8 | 1024 | 13 | 5 | 4 | 2 | 200 |

Liquid Z520 | 118 | 4 | 2015 | 5 | 32 | 8 | 1024 | 8 | 2 | 4 | 2 | 130 |

Liquid Z220 | 120 | 4 | 2015 | 4 | 32 | 8 | 1024 | 5 | 2 | 4 | 2 | 90 |

Liquid M220 | 119 | 4 | 2015 | 4 | 32 | 4 | 512 | 5 | 2 | 4 | 2 | 80 |

Liquid Z410 | 145 | 11 | 2015 | 4 | 32 | 8 | 1024 | 5 | 2 | 4 | 2 | 130 |

Liquid Jade S | 116 | 9 | 2014 | 5 | 32 | 16 | 2048 | 13 | 5 | 4 | 2 | 200 |

Liquid Z500 | 150 | 5 | 2014 | 5 | 32 | 4 | 1024 | 8 | 2 | 4 | 2 | 150 |

Liquid X1 | 164 | 5 | 2014 | 5 | 32 | 16 | 2048 | 13 | 2 | 4 | 2 | 260 |

Liquid Jade | 110 | 3 | 2014 | 5 | 32 | 8 | 1024 | 13 | 2 | 4 | 2 | 180 |

Liquid E700 | 155 | 5 | 2014 | 5 | 32 | 16 | 2048 | 8 | 2 | 4 | 2 | 200 |

Liquid E600 | 155 | 5 | 2014 | 5 | 32 | 4 | 1024 | 8 | 2 | 4 | 2 | 200 |

⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ |

${\mathit{A}}_{\mathit{i}}$ | ${\mathit{C}}_{1}$ | ${\mathit{C}}_{2}$ | ${\mathit{C}}_{3}$ | ${\mathit{C}}_{4}$ | ${\mathit{C}}_{5}$ | ${\mathit{C}}_{6}$ | ${\mathit{C}}_{7}$ | ${\mathit{C}}_{8}$ | ${\mathit{C}}_{9}$ | ${\mathit{C}}_{10}$ | ${\mathit{C}}_{11}$ | ${\mathit{C}}_{12}$ |
---|---|---|---|---|---|---|---|---|---|---|---|---|

${A}_{1}$ | 0.6959 | 0.4000 | 0.9995 | 0.5833 | 0.2500 | 0.1250 | 0.1250 | 0.5417 | 0.0005 | 1.0000 | 0.6667 | 0.9721 |

${A}_{2}$ | 0.8023 | 0.6667 | 0.9995 | 0.4167 | 0.5000 | 0.2500 | 0.1875 | 0.5417 | 0.0012 | 1.0000 | 0.6667 | 0.9590 |

${A}_{3}$ | 0.8058 | 0.6667 | 0.9990 | 0.4167 | 0.0625 | 0.2500 | 0.1875 | 0.5417 | 0.0032 | 1.0000 | 0.6667 | 0.9623 |

${A}_{4}$ | 0.8538 | 0.7333 | 0.9995 | 0.4167 | 0.0625 | 0.0625 | 0.0625 | 0.3333 | 0.0012 | 1.0000 | 0.6667 | 0.9820 |

${A}_{5}$ | 0.5871 | 0.2000 | 0.9990 | 0.6667 | 0.5000 | 0.2500 | 0.1250 | 0.2083 | 0.0005 | 1.0000 | 0.6667 | 0.9426 |

${A}_{6}$ | 0.8246 | 0.6667 | 0.9990 | 0.4167 | 0.5000 | 0.2500 | 0.1875 | 0.8750 | 0.0020 | 1.0000 | 1.0000 | 0.9639 |

${A}_{7}$ | 0.3684 | 0.9333 | 0.9990 | 0.8333 | 0.5000 | 0.1250 | 0.1250 | 0.2083 | 0.0005 | 1.0000 | 0.6667 | 0.9590 |

${A}_{8}$ | 0.4058 | 0.2000 | 0.9985 | 0.8333 | 0.5000 | 0.1250 | 0.0625 | 0.2083 | 0.0005 | 1.0000 | 0.6667 | 0.9688 |

${A}_{9}$ | 0.4058 | 0.2000 | 0.9985 | 0.8333 | 0.5000 | 0.2500 | 0.1250 | 0.2083 | 0.0005 | 1.0000 | 0.6667 | 0.9623 |

${A}_{10}$ | 0.8713 | 0.8000 | 0.9990 | 0.4167 | 0.0625 | 0.0625 | 0.0625 | 0.5417 | 0.0012 | 1.0000 | 0.6667 | 0.9672 |

${A}_{11}$ | 0.8620 | 0.7333 | 0.9990 | 0.4167 | 0.0625 | 0.0625 | 0.0625 | 0.3333 | 0.0005 | 1.0000 | 0.6667 | 0.9787 |

${A}_{12}$ | 0.8596 | 0.7333 | 0.9990 | 0.3333 | 0.0625 | 0.0625 | 0.0625 | 0.2083 | 0.0005 | 1.0000 | 0.6667 | 0.9852 |

${A}_{13}$ | 0.8608 | 0.7333 | 0.9990 | 0.3333 | 0.0625 | 0.0312 | 0.0312 | 0.2083 | 0.0005 | 1.0000 | 0.6667 | 0.9869 |

${A}_{14}$ | 0.8304 | 0.2667 | 0.9990 | 0.3333 | 0.0625 | 0.0625 | 0.0625 | 0.2083 | 0.0005 | 1.0000 | 0.6667 | 0.9787 |

${A}_{15}$ | 0.8643 | 0.4000 | 0.9985 | 0.4167 | 0.0625 | 0.1250 | 0.1250 | 0.5417 | 0.0012 | 1.0000 | 0.6667 | 0.9672 |

${A}_{16}$ | 0.8246 | 0.6667 | 0.9985 | 0.4167 | 0.0625 | 0.0312 | 0.0625 | 0.3333 | 0.0005 | 1.0000 | 0.6667 | 0.9754 |

${A}_{17}$ | 0.8082 | 0.6667 | 0.9985 | 0.4167 | 0.0625 | 0.1250 | 0.1250 | 0.5417 | 0.0005 | 1.0000 | 0.6667 | 0.9574 |

${A}_{18}$ | 0.8713 | 0.8000 | 0.9985 | 0.4167 | 0.0625 | 0.0625 | 0.0625 | 0.5417 | 0.0005 | 1.0000 | 0.6667 | 0.9705 |

${A}_{19}$ | 0.8187 | 0.6667 | 0.9985 | 0.4167 | 0.0625 | 0.1250 | 0.1250 | 0.3333 | 0.0005 | 1.0000 | 0.6667 | 0.9672 |

${A}_{20}$ | 0.8187 | 0.6667 | 0.9985 | 0.4167 | 0.0625 | 0.0312 | 0.0625 | 0.3333 | 0.0005 | 1.0000 | 0.6667 | 0.9672 |

⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ | ⋯ |

TOPSIS-COMET | COCOSO | EDAS | MAIRCA | MABAC | |||||
---|---|---|---|---|---|---|---|---|---|

Rank | Alternatives | Rank | Alternatives | Rank | Alternatives | Rank | Alternatives | Rank | Alternatives |

1.5 | ${A}_{120}$, ${A}_{613}$ | 1.5 | ${A}_{120}$, ${A}_{613}$ | 1.5 | ${A}_{59}$, ${A}_{552}$ | 1.5 | ${A}_{120}$, ${A}_{613}$ | 1.5 | ${A}_{120}$, ${A}_{613}$ |

3.5 | ${A}_{107}$, ${A}_{600}$ | 3.5 | ${A}_{591}$, ${A}_{98}$ | 3.5 | ${A}_{412}$, ${A}_{905}$ | 3.5 | ${A}_{591}$, ${A}_{98}$ | 3.5 | ${A}_{591}$, ${A}_{98}$ |

5.5 | ${A}_{260}$, ${A}_{753}$ | 5.5 | ${A}_{107}$, ${A}_{600}$ | 5.5 | ${A}_{413}$, ${A}_{906}$ | 5.5 | ${A}_{107}$, ${A}_{600}$ | 5.5 | ${A}_{107}$, ${A}_{600}$ |

7.5 | ${A}_{419}$, ${A}_{912}$ | 7.5 | ${A}_{305}$, ${A}_{798}$ | 7.5 | ${A}_{591}$, ${A}_{98}$ | 7.5 | ${A}_{305}$, ${A}_{798}$ | 7.5 | ${A}_{305}$, ${A}_{798}$ |

9.5 | ${A}_{591}$, ${A}_{98}$ | 9.5 | ${A}_{912}$, ${A}_{419}$ | 9.5 | ${A}_{276}$, ${A}_{769}$ | 9.5 | ${A}_{912}$, ${A}_{419}$ | 9.5 | ${A}_{912}$, ${A}_{419}$ |

11.5 | ${A}_{798}$, ${A}_{305}$ | 11.5 | ${A}_{412}$, ${A}_{905}$ | 11 | ${A}_{6}$ | 11.5 | ${A}_{412}$, ${A}_{905}$ | 11.5 | ${A}_{412}$, ${A}_{905}$ |

13 | ${A}_{6}$ | 13 | ${A}_{6}$ | 12.5 | ${A}_{273}$, ${A}_{766}$ | 13 | ${A}_{6}$ | 13 | ${A}_{6}$ |

14.5 | ${A}_{412}$, ${A}_{905}$ | 14.5 | ${A}_{800}$, ${A}_{307}$ | 14.5 | ${A}_{329}$, ${A}_{822}$ | 14.5 | ${A}_{800}$, ${A}_{307}$ | 14.5 | ${A}_{800}$, ${A}_{307}$ |

16.5 | ${A}_{307}$, ${A}_{800}$ | 16.5 | ${A}_{788}$, ${A}_{295}$ | 16.5 | ${A}_{772}$, ${A}_{279}$ | 16.5 | ${A}_{788}$, ${A}_{295}$ | 16.5 | ${A}_{788}$, ${A}_{295}$ |

18.5 | ${A}_{311}$, ${A}_{804}$ | 18.5 | ${A}_{311}$, ${A}_{804}$ | 18.5 | ${A}_{339}$, ${A}_{832}$ | 18.5 | ${A}_{311}$, ${A}_{804}$ | 18.5 | ${A}_{311}$, ${A}_{804}$ |

20.5 | ${A}_{315}$, ${A}_{808}$ | 20.5 | ${A}_{560}$, ${A}_{67}$ | 20.5 | ${A}_{804}$, ${A}_{311}$ | 20.5 | ${A}_{260}$, ${A}_{753}$ | 20.5 | ${A}_{260}$, ${A}_{753}$ |

… | … | … | … | … | … | … | … | … | … |

${\mathit{A}}_{\mathit{i}}$ | Copeland | Excluded Method Ranking | ||||
---|---|---|---|---|---|---|

Rank | $\mathbf{\text{TOPSIS-COMET}}$ | $\mathbf{COCOSO}$ | $\mathbf{EDAS}$ | $\mathbf{MAIRCA}$ | $\mathbf{MABAC}$ | |

${A}_{591}$ | 1 | 1 | 1 | 3 | 1 | 1 |

${A}_{98}$ | 2 | 2 | 2 | 5 | 2 | 2 |

${A}_{905}$ | 3 | 3 | 3 | 11 | 4 | 4 |

${A}_{412}$ | 4 | 4 | 4 | 12 | 3 | 3 |

${A}_{6}$ | 5 | 5 | 5 | 13 | 5 | 5 |

${A}_{804}$ | 6 | 7 | 6 | 21 | 6 | 6 |

${A}_{311}$ | 7 | 6 | 7 | 20 | 7 | 7 |

${A}_{906}$ | 8 | 8 | 8 | 22 | 8 | 8 |

${A}_{413}$ | 9 | 9 | 9 | 23 | 9 | 9 |

${A}_{766}$ | 10 | 10 | 11 | 29 | 11 | 11 |

${A}_{273}$ | 11 | 11 | 10 | 28 | 10 | 10 |

${A}_{613}$ | 12 | 20 | 18 | 1 | 19 | 19 |

${A}_{120}$ | 13 | 21 | 19 | 2 | 18 | 18 |

${A}_{558}$ | 14 | 12 | 14 | 38 | 14 | 14 |

${A}_{65}$ | 15 | 13 | 15 | 39 | 15 | 15 |

${A}_{808}$ | 16 | 16 | 12 | 36 | 13 | 13 |

${A}_{315}$ | 17 | 17 | 13 | 37 | 12 | 12 |

${A}_{305}$ | 18 | 24 | 26 | 8 | 26 | 26 |

${A}_{798}$ | 19 | 25 | 27 | 7 | 27 | 27 |

${A}_{419}$ | 20 | 31 | 30 | 9 | 30 | 30 |

… | … | … | … | … | … | … |

**Table 17.**Values of similarity coefficient ${r}_{w}$ for candidate compromise rankings with consideration of the whole dataset.

${\mathit{r}}_{\mathit{w}}$ | Method Rank | Mean | Min | ||||
---|---|---|---|---|---|---|---|

$\mathbf{\text{TOPSIS-COMET}}$ | $\mathbf{COCOSO}$ | $\mathbf{EDAS}$ | $\mathbf{MAIRCA}$ | $\mathbf{MABAC}$ | |||

Rank 1 | 0.992 | 0.9844 | 0.9389 | 0.9946 | 0.9946 | 0.9809 | 0.9389 |

Rank 2 | 0.9874 | 0.9845 | 0.9471 | 0.9912 | 0.9912 | 0.9803 | 0.9471 |

Rank 3 | 0.9915 | 0.9762 | 0.9446 | 0.9936 | 0.9936 | 0.9799 | 0.9446 |

Rank 4 | 0.997 | 0.9875 | 0.9044 | 0.9988 | 0.9988 | 0.9773 | 0.9044 |

Rank 5 | 0.9887 | 0.9841 | 0.9463 | 0.9915 | 0.9915 | 0.9804 | 0.9463 |

Rank 6 | 0.9887 | 0.9841 | 0.9463 | 0.9915 | 0.9915 | 0.9804 | 0.9463 |

**Table 18.**Values of similarity coefficient $WS$ for candidate compromise rankings with consideration of the whole dataset.

WS | Method Rank | Mean | Min | ||||
---|---|---|---|---|---|---|---|

$\mathbf{\text{TOPSIS-COMET}}$ | $\mathbf{COCOSO}$ | $\mathbf{EDAS}$ | $\mathbf{MAIRCA}$ | $\mathbf{MABAC}$ | |||

Rank 1 | 0.9914 | 0.9963 | 0.9949 | 0.9963 | 0.9963 | 0.99504 | 0.9914 |

Rank 2 | 0.9914 | 0.9963 | 0.9949 | 0.9963 | 0.9963 | 0.99504 | 0.9914 |

Rank 3 | 0.9914 | 0.9963 | 0.9949 | 0.9963 | 0.9963 | 0.99504 | 0.9914 |

Rank 4 | 0.9986 | 0.9994 | 0.8637 | 0.9994 | 0.9994 | 0.9721 | 0.8637 |

Rank 5 | 0.9914 | 0.9963 | 0.9949 | 0.9963 | 0.9963 | 0.99504 | 0.9914 |

Rank 6 | 0.9914 | 0.9963 | 0.9949 | 0.9963 | 0.9963 | 0.99504 | 0.9914 |

**Table 19.**Correlation matrix for ${r}_{w}$ values for candidate compromise rankings with consideration of the whole dataset.

${\mathit{r}}_{\mathit{w}}$ | Rank 1 | Rank 2 | Rank 3 | Rank 4 | Rank 5 | Rank 6 |
---|---|---|---|---|---|---|

Rank 1 | 1.0000 | 0.9994 | 0.9990 | 0.9957 | 0.9996 | 0.9996 |

Rank 2 | 0.9994 | 1.0000 | 0.9983 | 0.9929 | 0.9998 | 0.9998 |

Rank 3 | 0.9990 | 0.9983 | 1.0000 | 0.9931 | 0.9986 | 0.9986 |

Rank 4 | 0.9957 | 0.9929 | 0.9931 | 1.0000 | 0.9933 | 0.9933 |

Rank 5 | 0.9996 | 0.9998 | 0.9986 | 0.9933 | 1.0000 | 1.0000 |

Rank 6 | 0.9996 | 0.9998 | 0.9986 | 0.9933 | 1.0000 | 1.0000 |

**Table 20.**Correlation matrix for $WS$ values for candidate compromise rankings with consideration of the whole dataset.

WS | Rank 1 | Rank 2 | Rank 3 | Rank 4 | Rank 5 | Rank 6 |
---|---|---|---|---|---|---|

Rank 1 | 1.0000 | 1.0000 | 1.0000 | 0.9962 | 0.9998 | 0.9998 |

Rank 2 | 1.0000 | 1.0000 | 1.0000 | 0.9962 | 0.9998 | 0.9998 |

Rank 3 | 1.0000 | 1.0000 | 1.0000 | 0.9962 | 0.9998 | 0.9998 |

Rank 4 | 0.9893 | 0.9825 | 0.9843 | 1.0000 | 0.9841 | 0.9841 |

Rank 5 | 0.9998 | 0.9998 | 0.9998 | 0.9961 | 1.0000 | 1.0000 |

Rank 6 | 0.9998 | 0.9998 | 0.9998 | 0.9961 | 1.0000 | 1.0000 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Bączkiewicz, A.; Kizielewicz, B.; Shekhovtsov, A.; Wątróbski, J.; Sałabun, W.
Methodical Aspects of MCDM Based E-Commerce Recommender System. *J. Theor. Appl. Electron. Commer. Res.* **2021**, *16*, 2192-2229.
https://doi.org/10.3390/jtaer16060122

**AMA Style**

Bączkiewicz A, Kizielewicz B, Shekhovtsov A, Wątróbski J, Sałabun W.
Methodical Aspects of MCDM Based E-Commerce Recommender System. *Journal of Theoretical and Applied Electronic Commerce Research*. 2021; 16(6):2192-2229.
https://doi.org/10.3390/jtaer16060122

**Chicago/Turabian Style**

Bączkiewicz, Aleksandra, Bartłomiej Kizielewicz, Andrii Shekhovtsov, Jarosław Wątróbski, and Wojciech Sałabun.
2021. "Methodical Aspects of MCDM Based E-Commerce Recommender System" *Journal of Theoretical and Applied Electronic Commerce Research* 16, no. 6: 2192-2229.
https://doi.org/10.3390/jtaer16060122