Assessment of data quality and validity of composite innovation indicators with a neutrosophic multi-criteria approach
DOI: https://doi.org/10.3846/tede.2026.25283Abstract
In light of the emphasis on innovation as a driver of economic growth, new tools are needed to measure and compare national innovation systems. Improving methodological approaches to constructing composite innovation indicators also requires exploring issues related to aggregation, weighting, and reviewing the quality and validity of the data. This paper aims to support this debate by assessing the process and outcomes of innovation using fuzzy theory combined with a multicriteria technique to address the uncertainty attached to the underlying data used in constructing the composite indicator. Mainly, we deal with the problems related to the uniformity of the period covered by the indicator and the origin and reliability of the source of information by incorporating degrees of truth, indeterminacy, and falsity into the assessment process by applying neutrosophic numbers. To test the effectiveness of our approach, an empirical analysis is carried out based on the European Innovation Scoreboard and the assessment of the elementary criteria over the period 2020–2023.
Keywords:
innovation measurement, fuzzy sets, neutrosophic numbers, multicriteria decision making, data qualityHow to Cite
Share
License
Copyright (c) 2026 The Author(s). Published by Vilnius Gediminas Technical University.
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Abdel-Basset, M., Mohamed, M., Zhou, Y., & Hezam, I. (2017). Multi-criteria group decision making based on neutrosophic analytic hierarchy process. Journal of Intelligent and Fuzzy Systems, 33(6), 4055–4066. https://doi.org/10.3233/JIFS-17981
Abdel-Basset, M., Saleh, M., Gamal, A., & Smarandache, F. (2019). An approach of TOPSIS technique for developing supplier selection with group decision making under type-2 neutrosophic number. Applied Soft Computing Journal, 77, 438–452. https://doi.org/10.1016/j.asoc.2019.01.035
Abdel-Basset, M., Gamal, A., Chakrabortty, R. K., & Ryan, M. J. (2021). Evaluation approach for sustainable renewable energy systems under uncertain environment: A case study. Renewable Energy, 168, 1073–1095. https://doi.org/10.1016/j.renene.2020.12.124
Abdel-Basset, M., Mohamed, M., Mostafa, N. N., El-Henawy, I. M., & Abouhawwash, M. (2022). New multi-criteria decision-making technique based on neutrosophic axiomatic design. Scientific Reports, 12, Article 10657. https://doi.org/10.1038/s41598-022-14557-4
Alnafrah, I. (2021). Efficiency evaluation of BRICS’s national innovation systems based on bias-corrected network data envelopment analysis. Journal of Innovation and Entrepreneurship, 10, Article 26. https://doi.org/10.1186/s13731-021-00159-3
Arundel, A., & Hollanders, H. (2008). Innovation scoreboards: Indicators and policy use. (INNO-Metrics Thematic Paper). European Commission.
Atanassov, K. T. (1986). Intuitionistic fuzzy sets. Fuzzy Sets and Systems, 20(1), 87–96. https://doi.org/10.1016/S0165-0114(86)80034-3
Bano, S., Hasan, M. G., & Quddoos, A. (2023). A neutrosophic number based multi-choice best-worst multi-criteria decision-making approach and its applications. Industrial Engineering & Management Systems, 22(3), 224–243. https://doi.org/10.7232/iems.2023.22.3.224
Biswas, P., Pramanik, S., & Giri, B. C. (2018). TOPSIS strategy for multi-attribute decision making with trapezoidal neutrosophic numbers. Neutrosophic Sets and Systems, 19, 29–39. https://digitalrepository.unm.edu/nss_journal/vol19/iss1/6
Biswas, P., Pramanik, S., & Giri, B. C. (2019). Neutrosophic TOPSIS with group decision making. In C. Kahraman & I. Otay (Eds.), Fuzzy multi-criteria decision-making using neutrosophic sets (pp. 543–585). Springer. https://doi.org/10.1007/978-3-030-00045-5_21
Broumi, S., Nagarajan, D., Bakali, A., Talea, M., Smarandache, F., & Lathamaheswari, M. (2019). The shortest path problem in interval valued trapezoidal and triangular neutrosophic environment. Complex & Intelligent Systems, 5, 391–402. https://doi.org/10.1007/s40747-019-0092-5
Corrente, S., Garcia-Bernabeu, A., Greco, S., & Makkonen, T. (2023). Robust measurement of innovation performances in Europe with a hierarchy of interacting composite indicators. Economics of Innovation and New Technology, 32(2), 305–322. https://doi.org/10.1080/10438599.2021.1910815
Crespo, N. F., & Crespo, C. F. (2016). Global innovation index: Moving beyond the absolute value of ranking with a fuzzy-set analysis. Journal of Business Research, 69(11), 5265–5271. https://doi.org/10.1016/j.jbusres.2016.04.123
Deli, I., & Subas, Y. (2014). Single valued neutrosophic numbers and their applications to multicriteria decision making problem. Neutrosophic Set Systems, 2(1), 1–13.
Deli, I., & Şubaş, Y. (2017). A ranking method of single valued neutrosophic numbers and its applications to multi-attribute decision making problems. International Journal of Machine Learning and Cybernetics, 8, 1309–1322. https://doi.org/10.1007/s13042-016-0505-3
Edquist, C., Zabala-Iturriagagoitia, J. M., Barbero, J., & Zofío, J. L. (2018). On the meaning of innovation performance: Is the synthetic indicator of the Innovation Union Scoreboard flawed? Research Evaluation, 27(3), 196–211. https://doi.org/10.1093/reseval/rvy011
Elhassouny, A., & Smarandache, F. (2019). Neutrosophic modifications of Simplified TOPSIS for Imperfect Information (nS-TOPSIS). Neutrosophic Sets and Systems, 24, 100–114.
European Commission. (2023). European Innovation Scoreboard 2023. Publications Office of the European Union. https://data.europa.eu/doi/10.2777/119961
Fabri, S., Pace, L. A., Cassar, V., & Bezzina, F. (2023). Understanding the determinants of innovation across European member states: A fuzzy-set approach. International Journal of Innovation Science, 17(2), 356–372. https://doi.org/10.1108/IJIS-11-2022-0230
Freeman, C., & Soete, L. (2009). Developing science, technology and innovation indicators: What we can learn from the past. Research Policy, 38(4), 583–589. https://doi.org/10.1016/j.respol.2009.01.018
Freudenberg, M. (2003). Composite indicators of country performance: A critical assessment (OECD Science, Technology and Industry Working Papers 2003/16). https://doi.org/10.1787/405566708255
Garcia-Bernabeu, A., Cabello, J. M., & Ruiz, F. (2020). A multi-criteria reference point based approach for assessing regional innovation performance in Spain. Mathematics, 8(5), Article 797. https://doi.org/10.3390/math8050797
Garcia-Bernabeu, A., Cabello, J. M., & Ruiz, F. (2022). A reference point-based proposal to build regional quality of life composite indicators. Social Indicators Research, 164, 11–30. https://doi.org/10.1007/s11205-021-02818-0
Görçün, Ö. F. (2022). A novel integrated MCDM framework based on Type-2 neutrosophic fuzzy sets (T2NN) for the selection of proper Second-Hand chemical tankers. Transportation Research Part E: Logistics and Transportation Review, 163, Article 102765. https://doi.org/10.1016/j.tre.2022.102765
Greco, S., Ishizaka, A., Tasiou, M., & Torrisi, G. (2019). On the methodological framework of composite indices: A review of the issues of weighting, aggregation, and robustness. Social Indicators Research, 141, 61–94. https://doi.org/10.1007/s11205-017-1832-9
Grupp, H. (1998). Foundations of the economics of innovation: Theory, measurement and practice. Edward Elgar Publishing. https://doi.org/10.4337/9781035303649
Grupp, H., & Schubert, T. (2010). Review and new evidence on composite innovation indicators for evaluating national performance. Research Policy, 39(1), 67–78. https://doi.org/10.1016/j.respol.2009.10.002
Hollanders, H., & Janz, N. (2013). Scoreboards and indicator reports. In F. Gault (Ed.), Handbook of innovation indicators and measurement (pp. 279–298). Edward Elgar Publishing. https://doi.org/10.4337/9780857933652.00021
Hwang, C. L., & Yoon, K. (1981). Multiple attributes decision making: Methods and applications a state-of-the-art survey. Springer. https://doi.org/10.1007/978-3-642-48318-9
Junaid, M., Xue, Y., Syed, M. W., Li, J. Z., & Ziaullah, M. (2020). A neutrosophic ahp and topsis framework for supply chain risk assessment in automotive industry of Pakistan. Sustainability, 12(1), Article 154. https://doi.org/10.3390/su12010154
Karaaslan, F., & Hunu, F. (2020). Type-2 single-valued neutrosophic sets and their applications in multi-criteria group decision making based on TOPSIS method. Journal of Ambient Intelligence and Humanized Computing, 11, 4113–4132. https://doi.org/10.1007/s12652-020-01686-9
Karabašević, D., Stanujkić, D., Zavadskas, E. K., Stanimirović, P., Popović, G., Predić, B., & Ulutaş, A. (2020). A novel extension of the TOPSIS method adapted for the use of single-valued neutrosophic sets and hamming distance for e-commerce development strategies selection. Symmetry, 12(8), Article 1263. https://doi.org/10.3390/sym12081263
Khan, M., Son, L. H., Ali, M., Chau, H. T. M., Na, N. T. N., & Smarandache, F. (2018). Systematic review of decision making algorithms in extended neutrosophic sets. Symmetry, 10(8), Article 314. https://doi.org/10.3390/sym10080314
Khatter, K. (2020). Neutrosophic linear programming using possibilistic mean. Soft Computing, 24, 16847–16867. https://doi.org/10.1007/s00500-020-04980-y
Li, X., & Huang, X. (2019). The three-way decisions method based on theory of reliability with SV-triangular neutrosophic numbers. Symmetry, 11(7), Article 888. https://doi.org/10.3390/sym11070888
Mairesse, J., & Mohnen, P. (2010). Using innovation surveys for econometric analysis. In B. H. Hall & N. Rosenberg (Eds.), Handbook of the economics of innovation (Vol. 2, pp. 1129–1155). https://doi.org/10.1016/S0169-7218(10)02010-1
Munda, G., & Nardo, M. (2005). Constructing consistent composite indicators: The issue of weights. European Commission. https://publications.jrc.ec.europa.eu/repository/handle/JRC32434
Munda, G., & Nardo, M. (2009). Noncompensatory/nonlinear composite indicators for ranking countries: A defensible setting. Applied Economics, 41(12), 1513–1523. https://doi.org/10.1080/00036840601019364
Nǎdǎban, S., & Dzitac, S. (2016, May 10–14). Neutrosophic TOPSIS: A general view. In Proceedings of the 2016 6th International Conference on Computers Communications and Control (pp. 250–253). Oradea, Romania. IEEE. https://doi.org/10.1109/ICCCC.2016.7496769
Nafei, A. H., Javadpour, A., Nasseri, H., & Yuan, W. (2021). Optimized score function and its application in group multiattribute decision making based on fuzzy neutrosophic sets. International Journal of Intelligent Systems, 36(12), 7522–7543. https://doi.org/10.1002/int.22597
Nagalakshmi, T. (2023). A neutrosophic approach in the optimal solution of a neutrosophic fuzzy capital budgeting problem. International Journal of Mathematics in Operational Research, 24(4), 491–509. https://doi.org/10.1504/IJMOR.2023.130116
Nancy, & Garg, H. (2019). A novel divergence measure and its based TOPSIS method for multi criteria decision-making under single-valued neutrosophic environment. Journal of Intelligent and Fuzzy Systems, 36(1), 101–115. https://doi.org/10.3233/JIFS-18040
Organisation for Economic Co-operation and Development. (2002). Frascati manual 2002: Proposed standard practice for surveys on research and experimental development, the measurement of scientific and technological activities. OECD Publishing. https://doi.org/10.1787/9789264199040-en
Reig-Mullor, J., Garcia-Bernabeu, A., Pla-Santamaria, D., & Vercher-Ferrandiz, M. (2022). Evaluating ESG corporate performance using a new neutrosophic AHP-TOPSIS based approach. Technological and Economic Development of Economy, 28(5), 1242–1266. https://doi.org/10.3846/tede.2022.17004
Saisana, M., & Tarantola, S. (2002). State-of-the-art report on current methodologies and practices for composite indicator development. Joint Research Centre, European Commission.
Smarandache, F. (1999). A unifying field in logics, neutrosophy: Neutrosophic probability, set and logic. American Research Press.
Smarandache, F. (2006). Neutrosophic set – a generalization of the intuitionistic fuzzy set. In Proceedings of the 2006 IEEE International Conference on Granular Computing (pp. 38–42). Atlanta, GA, USA. IEEE. https://doi.org/10.1109/GRC.2006.1635754
Vázquez, M. L., & Torres, L. C. (2024). Integrating neutrosophic numbers in regression analysis for enhancing predictive modelling through uncertainty representation. Plithogenic Logic and Computation, 1, 120–126. https://doi.org/10.61356/j.plc.2024.1258
Wang, F. (2024). Novel score function and standard coefficient-based single-valued neutrosophic MCDM for live streaming sales. Information Sciences, 654, Article 119836. https://doi.org/10.1016/j.ins.2023.119836
Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8(3), 338–353. https://doi.org/10.1016/S0019-9958(65)90241-X
Zaied, A. N. H., Gamal, A., & Ismail, M. (2019). An integrated neutrosophic and TOPSIS for evaluating airline service quality. Neutrosophic Sets and Systems, 29, 30–39. https://doi.org/10.5281/zenodo.3514395
View article in other formats
CrossMark check
Published
Issue
Section
Copyright
Copyright (c) 2026 The Author(s). Published by Vilnius Gediminas Technical University.
License
This work is licensed under a Creative Commons Attribution 4.0 International License.