Drawbacks of demand accuracy assessment models on the example of slow-moving spare parts in civil aviation
DOI: https://doi.org/10.3846/aviation.2025.25363Abstract
Lots of researchers worldwide use a big variety of forecast models to predict demand. After running the forecast model, researcher always has a question if received prediction was accurate or not. To do so, a number of methods exist to assess model accuracy. Application of accuracy assessment models itself is not complex. The most difficult part for researcher: interpretation of the result and the understanding of information to take the right decisions. Companies who do demand forecast in 95% of cases use only one accuracy assessment method for their forecast model. In case, companies do it for fast moving items and the business doesn’t have any special requirement for the result level, it could be accepted. But in case slow-moving inventory is used and the company requires a certain service level, then there is a space for potential mistakes when running one model only. This work figures out the drawbacks of the current approaches towards forecast accuracy assessment of spare parts with little transaction history and proposes approaches to choose right accuracy assessment models. Experiment on data of existing company A that does aircraft maintenance was run to study the results of various forecast accuracy assessment models.
Keywords:
forecast error, mean absolute percentage error, material shortage, intermittent demand, sporadic consumptionHow to Cite
Share
License
Copyright (c) 2025 The Author(s). Published by Vilnius Gediminas Technical University.
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Ahlburg, D. (1992). A commentary on error measures: Error measures and choice of a forecast method. International Journal of Forecasting, 8(1), 99–111. https://doi.org/10.1016/0169-2070(92)90010-7
Byrne, R. (2012). Beyond traditional time-series: Using demand sensing to improve forecasts in volatile times. The Journal of Business Forecasting, 31(2).
Chen, C., Twycross, J., & Garibaldi, J. (2017). A new accuracy measure based on bounded relative error for time series forecasting. PloS One, 12(3), Article e0174202. https://doi.org/10.1371/journal.pone.0174202
Feng, Y.-W., Chen, J.-Y., Cheng, L., & Zhu, S.-P. (2021). Civil aircraft spare parts prediction and configuration management techniques: Review and prospect. Advances in Mechanical Engineering, 13(6), 1–17. https://doi.org/10.1177/16878140211026173
Hoover, J. (2006). Measuring forecast accuracy: Omissions in today’s forecasting engines and demand-planning software. Foresight: The International Journal of Applied Forecasting, 4, 32–35.
Hyndman, R., & Athanasopoulos, G. (2021). Forecasting: Principles and practice. O Texts.
Hyndman, R., & Koehler, A. (2006). Another look at measures of forecast accuracy. International Journal of Forecasting, 22(4), 679–688. https://doi.org/10.1016/j.ijforecast.2006.03.001
Kanhere, M. (2025). Commercial aircraft aftermarket part market report 2025 (Report ID CMR119923) (Global ed.). Cognitive Market Research & Consulting.
Kiefer, D., Bauer, M., & Grimm, F. (2021). Univariate time series forecasting: Machine learning prediction of the best suitable forecast model based on time series characteristics. In A. Zimmermann, R. J. Howlett, L. C. Jain, & R. Schmidt (Eds), Human Centred Intelligent Systems. KES-HCIS 2021. Smart Innovation, Systems and Technologies (Vol. 244, pp. 152–162). Springer. https://doi.org/10.1007/978-981-16-3264-8_15
Kim, S., & Kim, H. (2016). A new metric of absolute percentage error for intermittent demand forecasts. International Journal of Forecasting, 32(3), 669–679. https://doi.org/10.1016/j.ijforecast.2015.12.003
Kolassa, S., & Martin, R. (2011). Percentage error can ruin your day (and rolling the dice shows how). The International Journal of Applied Forecasting, 23, 21–27.
Koutsandreas, D., Spiliotis, E., Petropoulos, F., & Assimakopoulos, V. (2021). On the selection of forecasting accuracy measures. Journal of the Operational Research Society, 73(5), 937–954. https://doi.org/10.1080/01605682.2021.1892464
Larin, D. (2022). Optimizing the level of slow moving aviation spare parts inventory with economic order quantity model using bootstrap method. In I. Kabashkin, I. Yatskiv, & O. Prentkovskis (Eds), Reliability and Statistics in Transportation and Communication. RelStat 2021. Lecture Notes in Networks and Systems (Vol. 410, pp. 237–245). Springer. https://doi.org/10.1007/978-3-030-96196-1_21
Larin, D. (2021). Precising the accuracy of aviation spare parts forecast. In I. Kabashkin, I. Yatskiv, & O. Prentkovskis (Eds), Reliability and statistics in transportation and communication. RelStat 2020. Lecture Notes in Networks and Systems (Vol. 195, pp. 205–212). Springer. https://doi.org/10.1007/978-3-030-68476-1_19
Larin, D., & Tolujevs, J. (2020). Defing the proper model for aviation spare parts forecast. In Reliability and statistics in transportation and communication. Lecture Notes in Networks and Systems (pp. 71–79). Springer. https://doi.org/10.1007/978-3-030-44610-9_8
Luiz de Biazzi, J. (2019). Exponential smoothing for intermittent demand with demand basis updated more frequently than seasonality factors. Gestão & Produção, 26(1), Article e1297. https://doi.org/10.1590/0104-530x1297-19
Makridakis, S., & Hibon, M. (1979). Accuracy of forecasting: An empirical investigation. Journal of the Royal Statistical Society, 142, 97–145. https://doi.org/10.2307/2345077
Markidakis, S., & Hibon, M. (2000). The M3-competition: Results, conclusions and implications. International Journal of Forecasting, 16(4), 451–476. https://doi.org/10.1016/S0169-2070(00)00057-1
Melnikova, O. (2019). Crosoton’s method in the modification of Syntetos and Boylan for forecasting inventories. Scientific result. Economic Research, 5, 3–9. https://doi.org/10.18413/2409-1634-2019-5-2-0-1
Ord, K., Beaumont, A., & Snyder, R. (2010). Forecasting the intermittent demand (Working papers 2010-003). Research Program on Forecasting.
Ouabouch, L. (2015). Overview on supply chain resilience. Materials Management Review, July, 16–18.
Pan, X., & Montreuil, B. (2021). Assessing multi-purpose forecasting accuracy. In Proceedings of the 2021 IISE Annual Conference (pp. 728–733). ProQuest.
Sargent, R. G. (2013). An introduction to verification and validation of simulation models. In Proceedings of the 2013 Winter Simulation Conference (WSC) (pp. 321–327). IEEE. https://doi.org/10.1109/WSC.2013.6721430
Schlegel, G. L., & Trent, R. J. (2015). Supply chain risk management. CRC Press. https://doi.org/10.1201/b17531
Schuermann, T. (2014). Stress testing banks. International Journal of Forecasting, 30(3), 717–728. https://doi.org/10.1016/j.ijforecast.2013.10.003
Sen, E. G. (2023). Inventory management and plannning of slow-moving spare parts [Master Thesis, Eindhoven University of Logistics]. Eidhoven University of Technology.
Spithourakis, G., Petropoulus, F., Nikolopoulos, K., & Assimakopoulos, V. (2014). A systemic view of ADIDA framework. IMA Journal of Management Mathematics, 25(2), 125–137. https://doi.org/10.1093/imaman/dps031
St-Aubin, P., & Agard, B. (2022). Precision and reliability of forecast performance metrics. Forecasting, 4(4), 882–903. https://doi.org/10.3390/forecast4040048
Svetunkov, I., Kourentzes, N., & Ord, K. (2022). Complex exponential smoothing. Naval Research Logistics, 69(8), 1108–1123. https://doi.org/10.1002/nav.22074
Syntetos, A., & Boylan, J. (2001). On the bias of intermittent demand estimates. International Journal of Production Economics, 71(1–3), 457–466. https://doi.org/10.1016/S0925-5273(00)00143-2
Tayman, J., Swanson, D., & Barr, C. (1999). In search of the ideal measure of accuracy for subnational demographic forecasts. Population Research and Policy Review, 18(5), 387–410. https://doi.org/10.1023/A:1006317430570
Wallstrom, P., & Segerstedt, A. (2010). Evaluation of forecasting error measurements and techniques for intermittent demand. International Journal of Production Economics, 128(2), 625–636. https://doi.org/10.1016/j.ijpe.2010.07.013
Yapar, G. (2016). Modified simple exponential smoothing. Hacettepe Journal of Mathematics and Statistics, 47(143), 1–1. https://doi.org/10.15672/HJMS.201614320580
Zhu, Z. (2021). Spare parts demand forecasting and inventory management: Contributions to intermittent demand forecasting, installed base information and shutdown maintenance [Doctoral Thesis, Erasmus University Rotterdam]. Erasmus Universiteit Rotterdam (EUR).
View article in other formats
CrossMark check
Published
Issue
Section
Copyright
Copyright (c) 2025 The Author(s). Published by Vilnius Gediminas Technical University.
License
This work is licensed under a Creative Commons Attribution 4.0 International License.