A Modified Analytic Hierarchy Process Suitable for Online Survey Preference Elicitation
Abstract
:1. Introduction
2. Materials and Methods
2.1. AHP vs. MAHP
2.2. Survey Design and Implementation
3. Results
3.1. Demographics and Response Rates
3.2. Two Alternatives: Sparkling or Still Water?
3.3. Three Alternatives: Beef, Fish, or Vegetable Lasagna?
3.4. Four Alternatives: Dessert
3.5. Respondents’ Preferences for Each Method (a User Perspective)
4. Discussion
4.1. Advantages of MAHP over Traditional AHP for Online Surveys
4.2. Advantages of MAHP over Other Weighting Methods
4.3. Other Advantages of MAHP
4.4. Limitations of MAHP
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Gan, X.; Fernandez, I.C.; Guo, J.; Wilson, M.; Zhao, Y.; Zhou, B.; Wu, J. When to use what: Methods for weighting and aggregating sustainability indicators. Ecol. Indic. 2017, 81, 491–502. [Google Scholar] [CrossRef]
- Roberts, R.; Goodwin, P. Weight approximations in multi-attribute decision models. J. Multi-Criteria Decis. Anal. 2002, 11, 291–303. [Google Scholar] [CrossRef]
- Bottomley, P.A.; Doyle, J.R. A comparison of three weight elicitation methods: Good, better, and best. Omega 2001, 29, 553–560. [Google Scholar] [CrossRef]
- Bottomley, P.A.; Doyle, J.R.; Green, R.H. Testing the reliability of weight elicitation methods: Direct rating versus point allocation. J. Mark. Res. 2000, 37, 508–513. [Google Scholar] [CrossRef]
- Wang, J.-J.; Jing, Y.-Y.; Zhang, C.-F.; Zhao, J.-H. Review on multi-criteria decision analysis aid in sustainable energy decision-making. Renew. Sustain. Energy Rev. 2009, 13, 2263–2278. [Google Scholar] [CrossRef]
- Hayashi, K. Multicriteria analysis for agricultural resource management: A critical survey and future perspectives. Eur. J. Oper. Res. 2000, 122, 486–500. [Google Scholar] [CrossRef]
- Doyle, J.R.; Green, R.H.; Bottomley, P.A. Judging Relative Importance: Direct Rating and Point Allocation Are Not Equivalent. Organ. Behav. Hum. Decis. Process. 1997, 70, 65–72. [Google Scholar] [CrossRef] [PubMed]
- Aguarón, J.; Escobar, M.T.; Moreno-Jiménez, J.M. Reducing inconsistency measured by the geometric consistency index in the analytic hierarchy process. Eur. J. Oper. Res. 2021, 288, 576–583. [Google Scholar] [CrossRef]
- Cinelli, M.; Kadziński, M.; Gonzalez, M.; Słowiński, R. How to support the application of multiple criteria decision analysis? Let us start with a comprehensive taxonomy. Omega 2020, 96, 102261. [Google Scholar] [CrossRef]
- Pöyhönen, M.; Hämäläinen, R.P. On the convergence of multiattribute weighting methods. Eur. J. Oper. Res. 2001, 129, 569–585. [Google Scholar] [CrossRef]
- Van Ittersum, K.; Pennings, J.M.E.; Wansink, B.; van Trijp, H.C.M. The validity of attribute-importance measurement: A review. J. Bus. Res. 2007, 60, 1177–1190. [Google Scholar] [CrossRef]
- Saaty, T.L. The Analytic Hierarchy Process; McGraw-Hill: New York, NY, USA, 1980. [Google Scholar]
- Saaty, T.L. Decision-Making for Leaders; Wadsworth: Belmont, CA, USA, 1982. [Google Scholar]
- Canco, I.; Kruja, D.; Iancu, T. AHP, a reliable method for quality decision making: A case study in business. Sustainability 2021, 13, 13932. [Google Scholar] [CrossRef]
- Sequeira, M.; Hilletofth, P.; Adlemo, A. AHP-based support tools for initial screening of manufacturing reshoring decisions. J. Glob. Oper. Strateg. Sourc. 2021, 14, 502–527. [Google Scholar] [CrossRef]
- Rivero Gutiérrez, L.; De Vicente Oliva, M.A.; Romero-Ania, A. Managing Sustainable Urban Public Transport Systems: An AHP Multicriteria Decision Model. Sustainability 2021, 13, 4614. [Google Scholar] [CrossRef]
- Khan, I.; Pintelon, L.; Martin, H. The Application of Multicriteria Decision Analysis Methods in Health Care: A Literature Review. Med. Decis. Mak. 2022, 42, 262–274. [Google Scholar] [CrossRef]
- Deng, J.; King, B.; Bauer, T. Evaluating natural attractions for tourism. Ann. Tour. Res. 2002, 29, 422–438. [Google Scholar] [CrossRef]
- Hajkowicz, S.; Collins, K. A Review of Multiple Criteria Analysis for Water Resource Planning and Management. Water Resour. Manag. 2007, 21, 1553–1566. [Google Scholar] [CrossRef]
- Huang, I.B.; Keisler, J.; Linkov, I. Multi-criteria decision analysis in environmental sciences: Ten years of applications and trends. Sci. Total Environ. 2011, 409, 3578–3594. [Google Scholar] [CrossRef]
- Nielsen, J.R.; Mathiesen, C. Stakeholder preferences for Danish fisheries management of sand eel and Norway pout. Fish. Res. 2006, 77, 92–101. [Google Scholar] [CrossRef]
- Soma, K. How to involve stakeholders in fisheries management—A country case study in Trinidad and Tobago. Mar. Policy 2003, 27, 47–58. [Google Scholar] [CrossRef]
- Wattage, P.; Mardle, S. Stakeholder preferences towards conservation versus development for a wetland in Sri Lanka. J. Environ. Manag. 2005, 77, 122–132. [Google Scholar] [CrossRef]
- Baby, S. AHP Modeling for Multicriteria Decision-Making and to Optimise Strategies for Protecting Coastal Landscape Resources. Int. J. Innov. Manag. Technol. 2013, 4, 218–227. [Google Scholar] [CrossRef]
- Amor, S.B.; Belaid, F.; Benkraiem, R.; Ramdani, B.; Guesmi, K. Multi-criteria classification, sorting, and clustering: A bibliometric review and research agenda. Ann. Oper. Res. 2023, 325, 771–793. [Google Scholar] [CrossRef]
- Thadsin, K.; George, H.; Stanley, M. Introduction of AHP Satisfaction Index for workplace environments. J. Corp. Real Estate 2012, 14, 80–93. [Google Scholar]
- Bodin, L.; Gass, S.I. On teaching the analytic hierarchy process. Comput. Oper. Res. 2003, 30, 1487–1497. [Google Scholar] [CrossRef]
- Kwiesielewicz, M.; van Uden, E. Inconsistent and contradictory judgements in pairwise comparison method in the AHP. Comput. Oper. Res. 2004, 31, 713–719. [Google Scholar] [CrossRef]
- Lipovetsky, S.; Conklin, M.W. Robust estimation of priorities in the AHP. Eur. J. Oper. Res. 2002, 137, 110–122. [Google Scholar] [CrossRef]
- Danner, M.; Vennedey, V.; Hiligsmann, M.; Fauser, S.; Gross, C.; Stock, S. How Well Can Analytic Hierarchy Process be Used to Elicit Individual Preferences? Insights from a Survey in Patients Suffering from Age-Related Macular Degeneration. Patient-Patient-Cent. Outcomes Res. 2016, 9, 481–492. [Google Scholar] [CrossRef]
- Sato, Y.; Tan, K.H. Inconsistency indices in pairwise comparisons: An improvement of the Consistency Index. Ann. Oper. Res. 2023, 326, 809–830. [Google Scholar] [CrossRef]
- Bose, A. Improving consistency classification: An innovative benchmark-based approach for the AHP. J. Multi-Criteria Decis. Anal. 2024, 31, e1821. [Google Scholar] [CrossRef]
- Mufazzal, S.; Masood, S.; Khan, N.Z.; Muzakkir, S.M.; Khan, Z.A. Towards minimization of overall inconsistency involved in criteria weights for improved decision making. Appl. Soft Comput. 2021, 100, 106936. [Google Scholar] [CrossRef]
- Schmidt, F.L.; Hunter, J.E. Theory Testing and Measurement Error. Intelligence 1999, 27, 183–198. [Google Scholar] [CrossRef]
- Ishizaka, A.; Labib, A. Review of the main developments in the analytic hierarchy process. Expert Syst. Appl. 2011, 38, 14336–14345. [Google Scholar] [CrossRef]
- Tozer, P.R.; Stokes, J.R. Producer Breeding Objectives and Optimal Sire Selection. J. Dairy Sci. 2002, 85, 3518–3525. [Google Scholar] [CrossRef] [PubMed]
- Hummel, J.M.; Steuten, L.G.M.; Groothuis-Oudshoorn, C.J.M.; Mulder, N.; IJzerman, M.J. Preferences for Colorectal Cancer Screening Techniques and Intention to Attend: A Multi-Criteria Decision Analysis. Appl. Health Econ. Health Policy 2013, 11, 499–507. [Google Scholar] [CrossRef] [PubMed]
- Pascoe, S.; Dichmont, C.M.; Brooks, K.; Pears, R.; Jebreen, E. Management objectives of Queensland fisheries: Putting the horse before the cart. Mar. Policy 2013, 37, 115–122. [Google Scholar] [CrossRef]
- Ishizaka, A.; Lusti, M. An expert module to improve the consistency of AHP matrices. Int. Trans. Oper. Res. 2004, 11, 97–105. [Google Scholar] [CrossRef]
- Françozo, R.V.; Junior, L.S.V.U.; Carrapateira, E.S.; Pacheco, B.C.S.; Oliveira, M.T.; Torsoni, G.B.; Yari, J. A web-based software for group decision with analytic hierarchy process. MethodsX 2023, 11, 102277. [Google Scholar] [CrossRef] [PubMed]
- Yang, I.T.; Wang, W.-C.; Yang, T.-I. Automatic repair of inconsistent pairwise weighting matrices in analytic hierarchy process. Autom. Constr. 2012, 22 (Suppl. C), 290–297. [Google Scholar] [CrossRef]
- Lin, C.-C.; Wang, W.-C.; Yu, W.-D. Improving AHP for construction with an adaptive AHP approach (A3). Autom. Constr. 2008, 17, 180–187. [Google Scholar] [CrossRef]
- Karanik, M.; Wanderer, L.; Gomez-Ruiz, J.A.; Pelaez, J.I. Reconstruction methods for AHP pairwise matrices: How reliable are they? Appl. Math. Comput. 2016, 279 (Suppl. C), 103–124. [Google Scholar] [CrossRef]
- Benítez, J.; Delgado-Galván, X.; Izquierdo, J.; Pérez-García, R. Achieving matrix consistency in AHP through linearization. Appl. Math. Model. 2011, 35, 4449–4457. [Google Scholar] [CrossRef]
- Benítez, J.; Izquierdo, J.; Pérez-García, R.; Ramos-Martínez, E. A simple formula to find the closest consistent matrix to a reciprocal matrix. Appl. Math. Model. 2014, 38, 3968–3974. [Google Scholar] [CrossRef]
- Kou, G.; Ergu, D.; Shang, J. Enhancing data consistency in decision matrix: Adapting Hadamard model to mitigate judgment contradiction. Eur. J. Oper. Res. 2014, 236, 261–271. [Google Scholar] [CrossRef]
- Pascoe, S. A Simplified Algorithm for Dealing with Inconsistencies Using the Analytic Hierarchy Process. Algorithms 2022, 15, 442. [Google Scholar] [CrossRef]
- Pereira, V.; Costa, H.G. Nonlinear programming applied to the reduction of inconsistency in the AHP method. Ann. Oper. Res. 2015, 229, 635–655. [Google Scholar] [CrossRef]
- Cao, D.; Leung, L.C.; Law, J.S. Modifying inconsistent comparison matrix in analytic hierarchy process: A heuristic approach. Decis. Support Syst. 2008, 44, 944–953. [Google Scholar] [CrossRef]
- Pascoe, S.; Cannard, T.; Dowling, N.A.; Dichmont, C.M.; Breen, S.; Roberts, T.; Pears, R.J.; Leigh, G.M. Developing Harvest Strategies to Achieve Ecological, Economic and Social Sustainability in Multi-Sector Fisheries. Sustainability 2019, 11, 644. [Google Scholar] [CrossRef]
- Yang, Z.; Zhang, Z.-K.; Zhou, T. Anchoring bias in online voting. Europhys. Lett. 2012, 100, 68002. [Google Scholar] [CrossRef]
- Casper, W.C.; Edwards, B.D.; Wallace, J.C.; Landis, R.S.; Fife, D.A. Selecting response anchors with equal intervals for summated rating scales. J. Appl. Psychol. 2020, 105, 390–409. [Google Scholar] [CrossRef]
- Crawford, G.; Williams, C. A note on the analysis of subjective judgment matrices. J. Math. Psychol. 1985, 29, 387–405. [Google Scholar] [CrossRef]
- Aguarón, J.; Moreno-Jiménez, J.M. Local stability intervals in the analytic hierarchy process. Eur. J. Oper. Res. 2000, 125, 113–132. [Google Scholar] [CrossRef]
- Crawford, G.B. The geometric mean procedure for estimating the scale of a judgement matrix. Math. Model. 1987, 9, 327–334. [Google Scholar] [CrossRef]
- Aguarón, J.; Moreno-Jiménez, J.M. The geometric consistency index: Approximated thresholds. Eur. J. Oper. Res. 2003, 147, 137–145. [Google Scholar] [CrossRef]
- Zhang, R.; Gao, C.; Chen, X.; Li, F.; Yi, D.; Wu, Y. Genetic algorithm optimised Hadamard product method for inconsistency judgement matrix adjustment in AHP and automatic analysis system development. Expert Syst. Appl. 2023, 211, 118689. [Google Scholar] [CrossRef]
- Bose, A. Using genetic algorithm to improve consistency and retain authenticity in the analytic hierarchy process. Opsearch 2020, 57, 1070–1092. [Google Scholar] [CrossRef]
- Dhurkari, R.K. Remarks on the inconsistency measure of the Analytic Hierarchy Process. J. Intell. Fuzzy Syst. 2022, 43, 4669–4679. [Google Scholar] [CrossRef]
- Aloysius, J.A.; Davis, F.D.; Wilson, D.D.; Ross Taylor, A.; Kottemann, J.E. User acceptance of multi-criteria decision support systems: The impact of preference elicitation techniques. Eur. J. Oper. Res. 2006, 169, 273–285. [Google Scholar] [CrossRef]
- Hajkowicz, S.A.; McDonald, G.T.; Smith, P.N. An Evaluation of Multiple Objective Decision Support Weighting Techniques in Natural Resource Management. J. Environ. Plan. Manag. 2000, 43, 505–518. [Google Scholar] [CrossRef]
- Abastante, F.; Corrente, S.; Greco, S.; Ishizaka, A.; Lami, I.M. A new parsimonious AHP methodology: Assigning priorities to many objects by comparing pairwise few reference objects. Expert Syst. Appl. 2019, 127, 109–120. [Google Scholar] [CrossRef]
- Duleba, S. Introduction and comparative analysis of the multi-level parsimonious AHP methodology in a public transport development decision problem. J. Oper. Res. Soc. 2022, 73, 230–243. [Google Scholar] [CrossRef]
- Jamal Mahdi, A.; Esztergár-Kiss, D. Prioritizing Influential Factors on Tourist Decision-Making for Choosing Destinations Using Parsimonious Analytical Hierarchy Process (PAHP). Tour. Plan. Dev. 2024, 21, 330–353. [Google Scholar] [CrossRef]
- Stanco, M.; Lerro, M.; Marotta, G. Consumers’ Preferences for Wine Attributes: A Best-Worst Scaling Analysis. Sustainability 2020, 12, 2819. [Google Scholar] [CrossRef]
- Rezaei, J. Best-worst multi-criteria decision-making method. Omega 2015, 53, 49–57. [Google Scholar] [CrossRef]
- Rezaei, J. Best-worst multi-criteria decision-making method: Some properties and a linear model. Omega 2016, 64, 126–130. [Google Scholar] [CrossRef]
- Liu, P.; Zhu, B.; Wang, P. A weighting model based on best–worst method and its application for environmental performance evaluation. Appl. Soft Comput. 2021, 103, 107168. [Google Scholar] [CrossRef]
- Verzosa, N.; Greaves, S.; Ellison, R.; Ellison, A.; Davis, M. Eliciting preferences for ‘gamified’ travel surveys: A best-worst approach. Transp. Res. Procedia 2018, 32, 211–223. [Google Scholar] [CrossRef]
- Xu, R.H.; Zhou, L.-M.; Wong, E.L.-Y.; Wang, D. Investigating Medical Student’s Preferences for Internet-Based Healthcare Services: A Best-Worst Scaling Survey. Front. Public Health 2021, 9, 757310. [Google Scholar] [CrossRef]
- Schuster, A.L.R.; Crossnohere, N.L.; Campoamor, N.B.; Hollin, I.L.; Bridges, J.F.P. The rise of best-worst scaling for prioritization: A transdisciplinary literature review. J. Choice Model. 2024, 50, 100466. [Google Scholar] [CrossRef]
- Joshi, A.; Kale, S.; Chandel, S.; Pal, D.K. Likert Scale: Explored and Explained. Curr. J. Appl. Sci. Technol. 2015, 7, 396–403. [Google Scholar] [CrossRef]
- Kusmaryono, I.; Wijayanti, D.; Maharani, H.R. Number of Response Options, Reliability, Validity, and Potential Bias in the Use of the Likert Scale Education and Social Science Research: A Literature Review. Int. J. Educ. Methodol. 2022, 8, 625–637. [Google Scholar] [CrossRef]
- Wu, H.; Leung, S.-O. Can Likert Scales be Treated as Interval Scales?—A Simulation Study. J. Soc. Serv. Res. 2017, 43, 527–532. [Google Scholar] [CrossRef]
- Maleki, H.; Zahir, S. A Comprehensive Literature Review of the Rank Reversal Phenomenon in the Analytic Hierarchy Process. J. Multi-Criteria Decis. Anal. 2013, 20, 141–155. [Google Scholar] [CrossRef]
Alternative | Mean Score | Mean Difference | t-Statistic | Probability | |
---|---|---|---|---|---|
MAHP | AHP | ||||
Still | 0.800 | 0.836 | −0.036 | −0.790 | 0.440 |
Sparkling | 0.199 | 0.164 | 0.036 | 0.790 | 0.440 |
Alternative | Mean Score | Mean Difference | t-Statistic | Probability | |
---|---|---|---|---|---|
MAHP | AHP | ||||
All observations | |||||
Beef | 0.357 | 0.342 | 0.016 | 0.353 | 0.729 |
Fish | 0.371 | 0.329 | 0.042 | 0.973 | 0.349 |
Lasagna | 0.272 | 0.329 | −0.058 | −1.004 | 0.334 |
Consistent subset (AHP) | |||||
Beef | 0.336 | 0.271 | 0.064 | 1.205 | 0.262 |
Fish | 0.386 | 0.362 | 0.025 | 0.452 | 0.663 |
Lasagna | 0.278 | 0.367 | −0.089 | −1.019 | 0.338 |
Inconsistent subset (AHP) | |||||
Beef | 0.396 | 0.469 | −0.072 | −1.086 | 0.339 |
Fish | 0.343 | 0.270 | 0.073 | 0.960 | 0.392 |
Lasagna | 0.261 | 0.261 | −0.001 | −0.018 | 0.986 |
MAHP Rank | AHP Rank | ||
---|---|---|---|
1 | 2 | 3 | |
All observations | |||
1 | 14 | 6 | 5 |
2 | 7 | 15 | 3 |
3 | 4 | 4 | 17 |
Consistent subset | |||
1 | 5 | 3 | 1 |
2 | 4 | 5 | 0 |
3 | 0 | 1 | 8 |
Alternative | Mean Score | Mean Difference | t-Statistic | Probability | |
---|---|---|---|---|---|
MAHP | AHP | ||||
Ice cream | 0.244 | 0.276 | −0.032 | −0.943 | 0.359 |
Jelly | 0.129 | 0.094 | 0.035 | 2.954 | 0.009 |
Fruit | 0.435 | 0.498 | −0.062 | −2.197 | 0.042 |
Pie | 0.191 | 0.132 | 0.059 | 3.537 | 0.003 |
MAHP Rank | AHP Rank | |||
---|---|---|---|---|
1 | 2 | 3 | 4 | |
1 | 12 | 3 | 2 | 1 |
2 | 3 | 11 | 4 | 0 |
3 | 2 | 4 | 8 | 4 |
4 | 1 | 0 | 4 | 13 |
Alternative | Mean Score | Mean Difference | t-Statistic | Probability | |
---|---|---|---|---|---|
MAHP | AHP | ||||
MAHP | 0.557 | 0.492 | 0.065 | 0.774 | 0.449 |
AHP | 0.443 | 0.508 | −0.065 | −0.774 | 0.449 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Pascoe, S.; Farmery, A.; Nichols, R.; Lothian, S.; Azmi, K. A Modified Analytic Hierarchy Process Suitable for Online Survey Preference Elicitation. Algorithms 2024, 17, 245. https://doi.org/10.3390/a17060245
Pascoe S, Farmery A, Nichols R, Lothian S, Azmi K. A Modified Analytic Hierarchy Process Suitable for Online Survey Preference Elicitation. Algorithms. 2024; 17(6):245. https://doi.org/10.3390/a17060245
Chicago/Turabian StylePascoe, Sean, Anna Farmery, Rachel Nichols, Sarah Lothian, and Kamal Azmi. 2024. "A Modified Analytic Hierarchy Process Suitable for Online Survey Preference Elicitation" Algorithms 17, no. 6: 245. https://doi.org/10.3390/a17060245
APA StylePascoe, S., Farmery, A., Nichols, R., Lothian, S., & Azmi, K. (2024). A Modified Analytic Hierarchy Process Suitable for Online Survey Preference Elicitation. Algorithms, 17(6), 245. https://doi.org/10.3390/a17060245