Share:


The efficiency of machine learning algorithms in classifying non-functional requirements

    Milda Maciejauskaitė Affiliation
    ; Jolanta Miliauskaitė Affiliation

Abstract

Machine learning (ML) algorithms are more and more widely applied in various types of systems, so the research related to them is also increasing. One of the areas of research under consideration is the classification of non-functional requirements (NFRs) using ML algorithms. This area of research is important because the automatic classification of NFRs using high-performance ML algorithms and corresponding features helps requirements engineers classify non-functional requirements more accurately. This paper examines ML algorithms suitable for solving classification problems and their effectiveness in classifying non-functional requirements. Based on the described stages of the research methodology ML algorithms models were compared using the accuracy, precision, recall, and F-score metrics. A majority voting classifier model was created using Support Vector Machine, Naïve Bayes and K Nearest Neighbor Algorithm algorithms. After K-Fold cross validation were obtained these results: accuracy – 0.710 (scale from 0 to 1), precision – 0.845, recall – 0.814 and F-score – 0.815.

Keyword : machine learning, non-functional requirements, classification, support vector machine, ensemble models, K-Fold cross validation

How to Cite
Maciejauskaitė, M., & Miliauskaitė, J. (2024). The efficiency of machine learning algorithms in classifying non-functional requirements. New Trends in Computer Sciences, 2(1), 46–56. https://doi.org/10.3846/ntcs.2024.21574
Published in Issue
Jun 19, 2024
Abstract Views
51
PDF Downloads
47
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

References

Abad, Z. S., Karras, O., Ghazi, P., Glinz, M., Ruhe, G., & Schneider, K. (2017). What works better? A study of classifying requirements. In 2017 IEEE 25th International Requirements Engineering Conference (RE), (pp. 496–501). Lisbon. https://doi.org/10.1109/RE.2017.36

Alashqar, A. M. (2022). Studying the commonalities, mappings and relationships between non-functional requirements using machine learning. Science of Computer Programming, 218, Article 102806. https://doi.org/10.1016/j.scico.2022.102806

Bajaj, A. (2023, April 27). Ensemble models: How to make better predictions by combining multiple models with Python codes (explained). https://aryanbajaj13.medium.com/ensemble-models-how-to-make-better-predictions-by-combining-multiple-models-with-python-codes-6ac54403414e

Baker, C., Deng, L., Chakraborty, S., & Dehlinger, J. (2019). Automatic multi-class non-functional software requirements classification using neural networks. In 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC), (pp. 610–615). Milwaukee. https://doi.org/10.1109/COMPSAC.2019.10275

Bao, W., Lianju, N., & Yue, K. (2019). Integration of unsupervised and supervised machine learning algorithms for credit risk assessment. Expert Systems with Applications, 128, 301–315. https://doi.org/10.1016/j.eswa.2019.02.033

Binkhonain, M., & Zhao, L. (2019). A review of machine learning algorithms for identification and classification of non-functional requirements. Expert Systems with Applications. https://doi.org/10.1016/j.eswa.2019.02.031

Carta, S. (2022). What is machine learning? Wiley-Blackwell. https://doi.org/10.1002/9781119815075.ch18

Ghoneim, S. (2019, April 02). Accuracy, recall, precision, f-score & specificity, which to optimize on? https://medium.com/towards-data-science/accuracy-recall-precision-f-score-specificity-which-to-optimize-on-867d3f11124

Habibullah, K. M., & Horkoff, J. (2021, September). Non-functional requirements for machine learning: Understanding current use and challenges in industry. In 2021 IEEE 29th International Requirements Engineering Conference (RE) (pp. 13–23). IEEE. https://doi.org/10.1109/RE51729.2021.00009

Haque, M. A., Rahman, M. A., & Siddik, M. S. (2019). Non-functional requirements classification with feature extraction and machine learning: An empirical study. In 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT), (pp. 1–5). Dhaka, Bangladesh. https://doi.org/10.1109/ICASERT.2019.8934499

Harikrishnan, N. B. (2019, December 10). Confusion matrix, accuracy, precision, recall, F1 score. https://medium.com/analytics-vidhya/confusion-matrix-accuracy-precision-recall-f1-score-ade299cf63cd

Hendricks, R. (n.d.). What is a good accuracy score in Machine Learning? https://deepchecks.com/question/what-is-a-good-accuracy-score-in-machine-learning/

Ho, W. K., Tang, B.-S., & Wong, S. W. (2021). Predicting property prices with machine learning algorithms. Journal of Property Research, 38(1), 48–70. https://doi.org/10.1080/09599916.2020.1832558

Ibrahim, I. M., & Abdulazeez, A. M. (2021). The role of machine learning algorithms for diagnosing diseases. Journal of Applied Science and Technology Trends (JASTT), 2(1), 10–19. https://doi.org/10.38094/jastt20179

Imam, T., & Ananda, J. (2022). Machine learning for characterizing growth in tourism employment in developing economies: an assessment of tourism employment in Sri Lanka. Current Issues in Tourism, 25(16), 2695–2716. https://doi.org/10.1080/13683500.2021.1991895

Janiesch, C., Zschech, P., & Heinrich, K. (2021). Machine learning and deep learning. Electronic Markets, 31, 685–695. https://doi.org/10.1007/s12525-021-00475-2

Kanade, V. (2022). What is logistic regression? Equation, assumptions, types, and best practices. https://www.spiceworks.com/tech/artificial-intelligence/articles/what-is-logistic-regression/

Karthiban, R., Ambika, M., & Kannammal, K. E. (2019, January). A review on machine learning classification technique for bank loan approval. In 2019 International Conference on Computer Communication and Informatics (ICCCI) (pp. 1–6). IEEE. https://doi.org/10.1109/ICCCI.2019.8822014

Khurshid, I., Imtiaz, S., Boulila, W., Khan, Z., & Abbasi, A. (2022). Classification of non-functional requirements from IoT oriented healthcare requirement document. Frontiers Public Health, 10, Article 860536. https://doi.org/10.3389/fpubh.2022.860536

Koehrsen, W. (2018, March 03). Beyond accuracy: Precision and recall. https://medium.com/towards-data-science/beyond-accuracy-precision-and-recall-3da06bea9f6c

Kumar, R. (2023, August 12). VotingClassifier. https://medium.com/@ranjankumar_29097/votingclassifier-3f85ba8e4580

Kurtanović, Z., & Maalej, W. (2017). Automatically classifying functional and non-functional requirements using supervised machine learning. In 2017 IEEE 25th International Requirements Engineering Conference (RE), (pp. 490–495). Lisbon. https://doi.org/10.1109/RE.2017.82

Mahesh, B. (2020). Machine learning algorithms – A review. International Journal of Science and Research (IJSR), 9(1), 381–386.

Miller, M. I., Shih, L. C., & Kolachalama, V. B. (2023). Machine learning in clinical trials: A primer with applications to neurology. Neurotherapeutics, 20(4), 1066–1080. https://doi.org/10.1007/s13311-023-01384-2

Mohd, T., Masrom, S., & Johari, N. (2019). Machine learning housing price prediction in Petaling Jaya, Selangor, Malaysia. International Journal of Recent Technology and Engineering, 8(2S11), 542–546. https://doi.org/10.35940/ijrte.B1084.0982S1119

Nelson, D. (2020, October 26). Kas yra ansamblinis mokymasis? https://www.unite.ai/lt/kas-yra-ansamblinis-mokymasis/

Rajaguru, H., & Chakravarthy, S. (2019). Analysis of decision tree and K-Nearest neighbor algorithm in the classification of breast cancer. Asian Pacific Journal Cancer Prevention, 20(12), 3777–3781. https://doi.org/10.31557/APJCP.2019.20.12.3777

Rymarczyk, T., Kozłowski, E., Kłosowski, G., & Niderla, K. (2019). Logistic regression for machine learning in process tomography. Sensors, 19(15), Article 3400. https://doi.org/10.3390/s19153400

Sarker, I. H., Kayes, A. S., Badsha, S., Alqahtani, H., Watters, P., & Ng, A. (2020). Cybersecurity data science: An overview from machine learning perspective. Journal of Big Data, 7, Article 41. https://doi.org/10.1186/s40537-020-00318-5

Shukla, V. (2023, February). Software requirements dataset. https://www.kaggle.com/datasets/iamvaibhav100/software-requirements-dataset?resource=download

Shung, K. P. (2018, March 15). Accuracy, precision, recall or F1? https://medium.com/towards-data-science/accuracy-precision-recall-or-f1-331fb37c5cb9

Silwal, D. (2022, January 05). Confusion matrix, accuracy, precision, recall & F1 score: Interpretation of performance measures. https://www.linkedin.com/pulse/confusion-matrix-accuracy-precision-recall-f1-score-measures-silwal

Singh, A. (2023, November 22). A comprehensive guide to ensemble learning (with Python codes). https://www.analyticsvidhya.com/blog/2018/06/comprehensive-guide-for-ensemble-models/

Sruthi, E. R. (2023, April 26). Understand random forest algorithms with examples (updated 2023). https://www.analyticsvidhya.com/blog/2021/06/understanding-random-forest/

Uddin, S., Khan, A., Hossain, M. E., & Moni, M. A. (2019). Comparing different supervised machine learning algorithms for disease prediction. BMC Medical Informatics and Decision Making, 19, Article 281. https://doi.org/10.1186/s12911-019-1004-8

Wickramasinghe, I., & Kalutarage, H. (2021). Naive Bayes: Applications, variations and vulnerabilities: a review of literature with code snippets for implementation. Soft Computing, 25, 2277–2293. https://doi.org/10.1007/s00500-020-05297-6

Yang, L., & Shami, A. (2020). On hyperparameter optimization of machine learning algorithms: Theory and practice. Neurocomputing, 415, 295–316. https://doi.org/10.1016/j.neucom.2020.07.061