Cockpit crew safety performance prediction based on the integrated machine learning multi-class classification models and Markov chain

    Naimeh Borjalilu Affiliation
    ; Fariborz Jolai   Affiliation
    ; Mahdieh Tavakoli Affiliation


The main tool of cockpit crew performance evaluation is the recorded flight data used for flight operations safety improvement since all certified airlines require implementation of a safety and quality management system. The safety performance of a flight has been a challenging issue in the aviation industry and plays an important role to acquire competitive benefits. In this study, an integrated multi-class classification machine learning models and Markov chain were developed for cockpit crew performance evaluation during their flights. At the outset, the main features related to a flight are identified based on the literature review, flight operations expert’s statements, and the case study dataset (as numerical example). Afterwards, the flights’ performance is evaluated as a target column based on four multi-class classification models (Decision Tree, Support Vector Machine, Neural Network, and Random Forest). The results showed that the random forest classifier has the greatest value in all evaluation metrics (i.e., accuracy = 0.90, precision = 0.91, recall = 0.97, and F1-score = 0.93). Therefore, this model can be used by the airline companies to predict flight crew performance before the flight in order to prevent or decrease flight safety risks.

Keyword : cockpit crew performance, flight evaluation prediction, multi-class classification, Markov chain, safety management system

How to Cite
Borjalilu, N., Jolai, F., & Tavakoli, M. (2023). Cockpit crew safety performance prediction based on the integrated machine learning multi-class classification models and Markov chain . Aviation, 27(3), 152–161.
Published in Issue
Oct 18, 2023
Abstract Views
PDF Downloads
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.


Aljedani, N., Alotaibi, R., & Taileb, M. (2021). HMATC: Hierarchical multi-label Arabic text classification model using machine learning. Egyptian Informatics Journal, 22(3), 225–237.

Ashiku, L., Al-Amin, M., Madria, S., & Dagli, C. (2021). Machine learning models and big data tools for evaluating kidney acceptance. Procedia Computer Science, 185(June), 177–184.

Castillo-Botón, C., Casillas-Pérez, D., Casanova-Mateo, C., Ghimire, S., Cerro-Prada, E., Gutierrez, P. A., Deo, R. C., & Salcedo-Sanz, S. (2022). Machine learning regression and classification methods for fog events prediction. Atmospheric Research, 272, 106157.

Corker, K. M., & Pisanich, G. M. (1995). Analysis and modeling of flight crew performance in automated air traffic management systems. IFAC Proceedings Volumes, 28(15), 547–552.

Delgado, F., Trincado, R., & Pagnoncelli, B. K. (2019). A multistage stochastic programming model for the network air cargo allocation under capacity uncertainty. Transportation Research Part E: Logistics and Transportation Review, 131(November), 292–307.

EASA. (2019). AMC1 ORO.AOC.130 – Annex III.

European Aviation Safety Agency (EASA). (2016). Developing Standardised Fdm-Based Indicators Focus (2 December, pp. 1–55).

Filippone, A. (2008). Comprehensive analysis of transport aircraft flight performance. Progress in Aerospace Sciences, 44(3), 192–236.

Fodeh, S. J., & Tiwari, A. (2018). Exploiting MEDLINE for gene molecular function prediction via NMF based multi-label classification. Journal of Biomedical Informatics, 86(August 2017), 160–166.

Gharaibeh, A., Shaamala, A., Obeidat, R., & Al-Kofahi, S. (2020). Improving land-use change modeling by integrating ANN with Cellular Automata-Markov Chain model. Heliyon, 6(9), e05092.

Güven, İ., & Şimşir, F. (2020). Demand forecasting with color parameter in retail apparel industry using artificial neural networks (ANN) and support vector machines (SVM) methods. Computers and Industrial Engineering, 147.

Hon, K. K., Ng, C. W., & Chan, P. W. (2020). Machine learning based multi-index prediction of aviation turbulence over the Asia-Pacific. Machine Learning with Applications, 2, 100008.

International Civil Aviation Organization. (2013). Operation of Aircraft – Fatigue. Excerpts of Fatigue Management Realted Provisions from: Annex 6 to the Convention of International Civil Aviation, February.

Kulkarni, V. G. (2011). Brownian motion. In Introduction to modeling and analysis of stochastic systems (pp. 247–280). Springer.

Lan, C. E., Wu, K., & Yu, J. (2012). Flight characteristics analysis based on QAR data of a jet transport during landing at a high-altitude airport. Chinese Journal of Aeronautics, 25(1), 13–24.

Li, H., Wang, W., Fan, L., Li, Q., & Chen, X. (2020). A novel hybrid MCDM model for machine tool selection using fuzzy DEMATEL, entropy weighting and later defuzzification VIKOR. Applied Soft Computing, 91, 106207.

Lyu, Y., & Liem, R. P. (2020). Flight performance analysis with data-driven mission parameterization: Mapping flight operational data to aircraft performance analysis. Transportation Engineering, 2(September), 100035.

Martini, G., Scotti, D., & Volta, N. (2013). Including local air pollution in airport efficiency assessment: A hyperbolic-stochastic approach. Transportation Research Part D: Transport and Environment, 24(2007), 27–36.

Mokhtarimousavi, S., & Mehrabi, A. (2022). Flight delay causality: Machine learning technique in conjunction with random parameter statistical analysis. International Journal of Transportation Science and Technology, 12(1), 230–244.

Moral-García, S., Mantas, C. J., Castellano, J. G., & Abellán, J. (2020). Non-parametric predictive inference for solving multi-label classification. Applied Soft Computing Journal, 88.

Moshkov, M. (2021). On the depth of decision trees over infinite 1-homogeneous binary information systems. Array, 10(March), 100060.

Nguyen, T. P., & Lin, Y. K. (2021). Reliability assessment of a stochastic air transport network with late arrivals. Computers and Industrial Engineering, 151(January).

Okwuashi, O., & Ndehedehe, C. E. (2021). Integrating machine learning with Markov chain and cellular automata models for modelling urban land use change. Remote Sensing Applications: Society and Environment, 21(January).

Onan, A., Korukoğlu, S., & Bulut, H. (2016a). Ensemble of keyword extraction methods and classifiers in text classification. Expert Systems with Applications, 57, 232–247.

Onan, A., Bal, V., & Yanar Bayam, B. (2016b). The use of data mining for strategic management: A case study on mining association rules in student information system. Croatian Journal of Education: Hrvatski časopis za odgoj i obrazovanje, 18(1), 41–70.

Onan, A. (2019). Two-stage topic extraction model for bibliometric data analysis based on word embeddings and clustering. IEEE Access, 7, 145614–145633.

Onan, A., & Korukoğlu, S. (2017). A feature selection model based on genetic rank aggregation for text sentiment classification. Journal of Information Science, 43(1), 25–38.

Onan, A. (2021). Sentiment analysis on product reviews based on weighted word embeddings and deep neural networks. Concurrency and Computation: Practice and Experience, 33(23), e5909.

Onan, A. (2015). A fuzzy-rough nearest neighbor classifier combined with consistency-based subset evaluation and instance selection for automated diagnosis of breast cancer. Expert Systems with Applications, 42(20), 6844–6852.

Onan, A. (2020). Mining opinions from instructor evaluation reviews: A deep learning approach. Computer Applications in Engineering Education, 28(1), 117–138.

Onan, A. (2016). Classifier and feature set ensembles for web page classification. Journal of Information Science, 42(2), 150–165.

Onan, A. (2018a). An ensemble scheme based on language function analysis and feature engineering for text genre classification. Journal of Information Science, 44(1), 28–47.

Onan, A. (2018b). Biomedical text categorization based on ensemble pruning and optimized topic modelling. Computational and Mathematical Methods in Medicine, 2018.

Onan, A. (2019a). Consensus clustering-based undersampling approach to imbalanced learning. Scientific Programming, 2019.

Onan, A. (2019b). Topic-enriched word embeddings for sarcasm identification. In R. Silhavy, Software Engineering Methods in Intelligent Algorithms: Proceedings of 8th Computer Science On-line Conference 2019 (Vol. 984, pp. 293–304). Springer International Publishing.

Onan, A., & Toçoğlu, M. A. (2021). A term weighted neural language model and stacked bidirectional LSTM based framework for sarcasm identification. IEEE Access, 9, 7701–7722.

Oreschko, B., Kunze, T., Schultz, M., Fricke, H., Kumar, V., & Sherry, L. (2012). Turnaround prediction with stochastic process times and airport specific delay pattern airport delays. In The 5th International Conference on Research in Air Transportation, 9. ResearchGate.

Papadopoulos, C. T., Li, J., & O’Kelly, M. E. J. (2019). A classification and review of timed Markov models of manufacturing systems. Computers and Industrial Engineering, 128(November 2018), 219–244.

Qian, W., Xiong, C., & Wang, Y. (2021). A ranking-based feature selection for multi-label classification with fuzzy relative discernibility. Applied Soft Computing, 102.

Rey, M., Aloise, D., Soumis, F., & Pieugueu, R. (2021). A data-driven model for safety risk identification from flight data analysis. Transportation Engineering, 5, 100087.

Ross, S. M. (2014). Introduction to probability models. Academic Press.

Samaee, S., & Kobravi, H. R. (2020). Predicting the occurrence of wrist tremor based on electromyography using a hidden Markov model and entropy based learning algorithm. Biomedical Signal Processing and Control, 57(March).

Shone, R., Glazebrook, K., & Zografos, K. G. (2021). Applications of stochastic modeling in air traffic management: Methods, challenges and opportunities for solving air traffic problems under uncertainty. European Journal of Operational Research, 292(1), 1–26.

Toçoğlu, M. A., & Onan, A. (2020, July). Sentiment analysis on students’ evaluation of higher educational institutions. In International Conference on Intelligent and Fuzzy Systems (pp. 1693–1700). INFUS 2020: Intelligent and Fuzzy Techniques: Smart and Innovative Solutions. Springer.

Utami, N. A., Maharani, W., & Atastina, I. (2021). Personality classification of Facebook users according to big five personality using SVM (Support Vector Machine) method. Procedia Computer Science, 179(2020), 177–184.

Van Giffen, B., Herhausen, D., & Fahse, T. (2022). Overcoming the pitfalls and perils of algorithms: A classification of machine learning biases and mitigation methods. Journal of Business Research, 144, 93–106.

Wang, L., Wu, C., & Sun, R. (2014). An analysis of flight Quick Access Recorder (QAR) data and its applications in preventing landing incidents. Reliability Engineering and System Safety, 127, 86–96.

Yaakoubi, Y., Soumis, F., & Lacoste-Julien, S. (2020). Machine learning in airline crew pairing to construct initial clusters for dynamic constraint aggregation. EURO Journal on Transportation and Logistics, 9(4), 100020.

Yamaguchi, S., Nakashima, H., Moriwaki, Y., Terada, T., & Shimizu, K. (2022). Prediction of protein mononucleotide binding sites using AlphaFold2 and machine learning. Computational Biology and Chemistry, 107744.

Yan, S., & Tang, C.-H. (2007). A heuristic approach for airport gate assignments for stochastic flight delays. European Journal of Operational Research, 180(2), 547–567.

Yang, C., Yin, T., Zhao, W., Huang, D., & Fu, S. (2014). Human factors quantification via boundary identification of flight performance margin. Chinese Journal of Aeronautics, 27(4), 977–985.

Zhou, Y., Liu, Y., Wang, D., Liu, X., & Wang, Y. (2021). A review on global solar radiation prediction with machine learning models in a comprehensive perspective. Energy Conversion and Management, 235(13), 113960.