Improving AHP consistency through cognitive collaboration with large language models
DOI: https://doi.org/10.3846/ntcs.2025.25442Abstract
The Analytic Hierarchy Process (AHP) is a human-centered method designed to structure complex problems and extract the authentic, consistent opinions of decision-makers. However, its practical application is often limited by inconsistency in human judgments, often caused by the respondent’s insufficient understanding of the task rather than simple mathematical error. The main goal of the article is to explore the possibilities of integration of innovative Artificial Intelligence (AI) tools for improving the AHP method. In order to improve respondent understanding and facilitate more intuitive and transparent consistency adjustments, this study also analyzes how to reduce the occurrence of inconsistency in pairwise comparison matrices and improve the Consistency Ratio (CR) by using advanced capabilities of large language models. The initial stage of study included a literature review, identifying typical problems in this area, reviewing the tools and methods used for obtaining better results, and presenting areas for improvement. At the second stage, the possibilities for improving consistency by increasing the influence of humans as decision makers, moving from the use of powerful mathematical optimization mechanisms to the application of human-centered explanatory AI techniques were analyzed. Based on the study results, the description of approaches for improvement of consistency in AHP was presented.
First published online 27 January 2026
Keywords:
analytic hierarchy process (AHP), large language models (LLM), multi-criteria decision making (MCDM), human-centered AI, explainable AI (XAI), consistency management, decision support systems, consistency ratioHow to Cite
Share
License
Copyright (c) 2025 The Author(s). Published by Vilnius Gediminas Technical University.
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Bauer, J. M., & Michalowski, M. (2025). Human-centered explainability evaluation in clinical decision-making: A critical review of the literature. Journal of the American Medical Informatics Association, 32(9), 1477–1484. https://doi.org/10.1093/jamia/ocaf110
Bose, A. (2023). Improving consistency classification: An innovative benchmark‐based approach for the AHP. Journal of Multi-Criteria Decision Analysis, 31(1). https://doi.org/10.1002/mcda.1821
Cheng, F., Li, H., Liu, F., van Rooij, R., Zhang, K., & Lin, Z. (2025). Empowering LLMs with logical reasoning: A comprehensive survey. In The 34th International Joint Conference on Artificial Intelligence (IJCAI). https://doi.org/10.24963/ijcai.2025/1155
Çoban, V. (2023). Developing random indices and consistency ratios for inconsistency methods in pairwise comparison. Journal of The Faculty of Engineering and Architecture of Gazi University, 38(2), 781–793. https://doi.org/10.17341/gazimmfd.903495
Čančer, V. (2024). Selection procedure of the approximation methods for deriving priorities: A case of inconsistent pairwise comparisons. Business Systems Research Journal, 15(2), 21–30. https://doi.org/10.2478/bsrj-2024-0015
Ding, S., Pan, X., Hu, L., & Liu, L. (2025). A new model for calculating human trust behavior during human-AI collaboration in multiple decision-making tasks: A Bayesian approach. Computers & Industrial Engineering, 200, Article 110872. https://doi.org/10.1016/j.cie.2025.110872
Elangovan, A., Liu, L., Xu, L., Bodapati, S. B., & Roth, D. (2024). ConSiDERS-the-human evaluation framework: Rethinking human evaluation for generative large language models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 1, 1137–1160. https://doi.org/10.18653/v1/2024.acl-long.63
Escobar, M. T., Aguarón, J., Moreno-Jiménez, J. M., & Turón, A. (2023). A decision support system for improving the inconsistency in AHP. International Journal of Decision Support System Technology, 15(2), 1–16. https://doi.org/10.4018/IJDSST.315644
Hassan, R., Nguyen, N., Finserås, S. R., Adde, L., Strümke, I., & Støen, R. (2025). Unlocking the black box: Enhancing human-AI collaboration in high-stakes healthcare scenarios through explainable AI. Technological Forecasting and Social Change, 219, Article 124265. https://doi.org/10.1016/j.techfore.2025.124265
Heymann, M. C., Pereira, V., & Caiado, R. G. G. (2024). PyMissingAHP: An evolutionary algorithm for filling missing values in incomplete pairwise comparisons matrices with real or fuzzy numbers via mono and multiobjective approaches. Arabian Journal for Science and Engineering, 49(1), 7375–7394. https://doi.org/10.1007/s13369-023-08227-4
Islam, R., Anis, A., & Azam, M. S. E. (2024). Revitalizing university performance evaluation: The case of SETARA model in Malaysia. Journal of Applied Research in Higher Education, 17(4), 1294–1312. https://doi.org/10.1108/JARHE-12-2023-0561
Kaushik, S., Pant, S., Joshi, L. K., Kumar A., & Ram, M. (2024). A review based on various applications to find a consistent pairwise comparison matrix. Journal of Reliability and Statistical Studies, 17(1), 45–76. https://doi.org/10.13052/jrss0974-8024.1713
Kim, J., Maathuis, H., & Sent, D. (2024). Human-centered evaluation of explainable AI applications: A systematic review. Frontiers in Artificial Intelligence, 7, Article 1456486. https://doi.org/10.3389/frai.2024.1456486
Kuraś, P., Strzałka, D., Kowal, B., Organiściak, P., Demidowski, K., & Vanivska, V. (2024). REDUCE-a tool supporting inconsistencies reduction in the decision-making process. Applied Sciences, 14(23), Article 11465. https://doi.org/10.3390/app142311465
Kuraś, P., Strzałka, D., Kowal, B., & Mazurek, J. (2023). REDUCE – a Python module for reducing inconsistency in pairwise comparison matrices. Advances in Science and Technology Research Journal, 17(4), 227–234. https://doi.org/10.12913/22998624/170187
Li, H., Dai, X., Wu, Q., Zhou, L., & Pedrycz, W. (2025). A Bayesian analysis framework for decision making with interval pairwise comparison judgments. Decision Analysis. https://doi.org/10.1287/deca.2024.0207
Lin, J., Tomlin, N., Andreas, J., & Eisner, J. (2024). Decision-oriented dialogue for human-AI collaboration. Transactions of the Association for Computational Linguistics, 12, 892–911. https://doi.org/10.1162/tacl_a_00679
Liu, F., Liu, T., & Hu, Y.-K. (2023a). Reaching consensus in group decision making with non-reciprocal pairwise comparison matrices. Applied Intelligence, 53, 12888–12907. https://doi.org/10.1007/s10489-022-04136-5
Liu, Z.-L., Liu, F., Zhang, J.-W., & Pedrycz, W. (2023b). Optimizing consistency and consensus in group decision making based on relative projection between multiplicative reciprocal matrices. Expert Systems with Applications, 224, Article 119948. https://doi.org/10.1016/j.eswa.2023.119948
Lou, B., Lu, T., Raghu, T. S., & Zhang, Y. (2025). Unraveling human-AI teaming: A review and outlook. SSRN. https://doi.org/10.2139/ssrn.5211067
Maceika, A., Bugajev, A., Šostak, O. R., & Vilutienė, T. (2021). Decision tree and AHP methods application for projects assessment: A case study. Sustainability, 13(10), Article 5502. https://doi.org/10.3390/su13105502
Morandini, S., Fraboni, F., Puzzo, G., Giusino, D., Volpi, L., Brendel, H., Balatti, E., De Angelis, M., De Cesarei A., & Pietrantoni, L. (2023). Examining the nexus between explainability of AI systems and user’s trust: A preliminary scoping review. CEUR Workshop Proceedings, 3554, 30–35. https://ceur-ws.org/Vol-3554/paper6.pdf
Mostafa, A. M. (2024). A group multi-criteria decision-making approach based on the best-only method for cloud service selection. IEEE Access, 12, 119946–119957. https://doi.org/10.1109/ACCESS.2024.3450280
Pan, Z., Moore, O. A., Papadimitriou, A., & Zhu, J. (2025). AI literacy and trust: A multi-method study of human-GAI team collaboration. Computers in Human Behavior: Artificial Humans, 4, Article 100162. https://doi.org/10.1016/j.chbah.2025.100162
Pant, S., Kumar, A., & Mazurek, J. (2025). An overview and comparison of axiomatization structures regarding inconsistency indices’ properties in pairwise comparisons methods: A decade of advancements. International Journal of Mathematical, Engineering and Management Sciences, 10(1), 265–284. https://doi.org/10.33889/IJMEMS.2025.10.1.015
Peters, U., & Chin-Yee, B. (2025). Generalization bias in large language model summarization of scientific research. Royal Society Open Science, 12, Article 241776. https://doi.org/10.1098/rsos.241776
Perez, J., & Ong, E. (2024). Designing an LLM-based dialogue tutoring system for novice programming. In Proceedings of the 32nd International Conference on Computers in Education. Asia-Pacific Society for Computers in Education. https://doi.org/10.58459/icce.2024.4954
Saaty, T. L. (2000). Fundamentals of decision making and priority theory with the Analytic Hierarchy Process (Analytic Hierarchy Process Series, Vol. 6). RWS Publications. https://doi.org/10.13033/isahp.y1999.038
Sato, Y., & Tan, K., H. (2023). Inconsistency indices in pairwise comparisons: An improvement of the consistency index. Annals of Operations Research, 326, 809–830. https://doi.org/10.1007/s10479-021-04431-3
Sowa, K., Przegalinska, A., & Ciechanowski, L. (2021). Cobots in knowledge work: Human-AI collaboration in managerial professions. Journal of Business Research, 125, 135–142. https://doi.org/10.1016/j.jbusres.2020.11.038
Srđević, B., & Srđević, Z. (2024). Multi-model assessing and visualizing consistency and compatibility of experts in group decision-making. Mathematics, 12, Article 1699. https://doi.org/10.3390/math12111699
Sun, Y., Zhuang, L., Jis, T., Cheng, D., Zhao, X., & Guo, J. (2025). A risk assessment method for power internet of things information security based on multi‐objective hierarchical optimisation. IET Smart Grid, 8(1), Article e12208. https://doi.org/10.1049/stg2.12208
Tiwari, R. (2023). Explainable AI (XAI) and its applications in building trust and understanding in AI decision making. International Journal of Scientific Research in Engineering and Management (IJSREM), 7(1), 1–13. https://doi.org/10.55041/IJSREM17592
Tong, X., & Wang, Z. J. (2023). New additive-consistency-driven methods for deriving two types of normalized utility vectors from additive reciprocal preference relations. Journal of the Operational Research Society, 74(6), 1475–1494. https://doi.org/10.1080/01605682.2022.2096503
Tu, J., Wu, Z., & Xu, J. (2023). Geometric consistency index for interval pairwise comparison matrices. Journal of the Operational Research Society, 74(5), 1229–1241. https://doi.org/10.1080/01605682.2022.2075803
Van Leersum, C. M., & Maathuis, C. (2025). Human centred explainable AI decision-making in healthcare. Journal of Responsible Technology, 21, Article 100108. https://doi.org/10.1016/j.jrt.2025.100108
Vommi, A. M., & Vommi, V. B. (2025). A novel scale for inconsistency reduction in the pair-wise comparison matrices. Foundations of Computing and Decision Sciences, 50(1), 87–114. https://doi.org/10.2478/fcds-2025-0004
Wang, P., Liu, Y., & Zhou, H. (2023). Research on the eco-geological environment carrying capacity in Pingwu County after the Wenchuan earthquake based on the modified AHP. Natural Hazards, 115, 2097–2115. https://doi.org/10.1007/s11069-022-05629-9
Wang, X. (2025). Human-centered evaluation and design of AI explanation in AI-assisted decision making (Doctoral dissertation). Purdue University, West Lafayette, Indiana, USA. https://doi.org/10.1145/3640544.3645239
Wang, Y., Zhou, L., Li, H., & Dai, X. (2024). Probabilistic consistency of stochastic multiplicative comparison matrices based on Monte Carlo simulation. Information Sciences, 656, Article 119896. https://doi.org/10.1016/j.ins.2023.119896
Xiao, J., & Wang, X. (2024). An optimization method for handling incomplete and conflicting opinions in quality function deployment based on consistency and consensus reaching process. Computers & Industrial Engineering, 187, Article 109779. https://doi.org/10.1016/j.cie.2023.109779
Xu, Y., Gao, W., Wang, Y., Shan, X., & Lin, Y.-S. (2024). Enhancing user experience and trust in advanced LLM-based conversational agents. Computing and Artificial Intelligence, 2(2), Article 1467. https://doi.org/10.59400/cai.v2i2.1467
Yuen, K. K. F. (2024). Closed-form solutions of consistency ratio in best worst method minmax optimization model: Max of edge error matrix and minmax edge error determinant methods. Granular Computing, 9, Article 42. https://doi.org/10.1007/s41066-024-00459-5
View article in other formats
CrossMark check
Published
Issue
Section
Copyright
Copyright (c) 2025 The Author(s). Published by Vilnius Gediminas Technical University.
License
This work is licensed under a Creative Commons Attribution 4.0 International License.