A method for detecting DDoS attacks in VoIP systems based on machine learning

Viktor Gnatyuk; Ivan Gorbachov

Journal: Volume 30, No. 3, 2025

Pages: 37 – 46

DOI: https://doi.org/10.62660/bcstu/3.2025.37

869 Views

A method for detecting DDoS attacks in VoIP systems based on machine learning

Viktor Gnatyuk, Ivan Gorbachov

Received 24.04.2025

Revised 05.08.2025

Accepted 15.09.2025

Abstract

Protecting VoIP systems from DDoS attacks is a critical issue, as such attacks can lead to significant financial losses and a decline in the quality of service for users. Existing methods of detecting attacks are based on signature analysis or traditional rules, which limits their effectiveness in cases of new or modified attacks. The aim of this work was to develop a method for detecting DDoS attacks in VoIP systems based on machine learning, which provides high accuracy in classifying abnormal traffic. To achieve this goal, methods of network traffic analysis, machine learning, and statistical evaluation of model effectiveness were used. The main research tool was a multilayer perceptron neural network trained on real network traffic. As a result of this research, a model was developed and tested that demonstrated high accuracy in detecting attacks. A comparative analysis of the effectiveness of the developed model with other approaches was carried out. The proposed method was integrated into the Asterisk environment through the Asterisk Manager Interface, which made it possible to monitor SIP traffic in real time, analyse it using a trained model, and automatically block attacking IP addresses through IPTables or Fail2Ban. Based on the results of the model comparison by metrics, the best model was selected and an algorithm for protecting VoIP from DDoS was developed based on it. The practical value of the work lies in the development of an effective method for protecting VoIP systems, which can be used to improve the level of security in telecommunications networks. The proposed approach can be scaled and adapted to different network infrastructure configurations

Keywords

IP telephony; SIP; Asterisk; neural network; cybersecurity

References

Chornobuk, M., Dubrovin, V., & Deineha, L. (2023). Cybersecurity: Research on methods for detecting DdoS attacks. Computer Systems and Information Technologies, 4, 6-9. doi: 10.31891/csit-2023-4-1 .
Cui, Y., Qian, Q., Guo, C., Shen, G., Tian, Y., Xing, H., & Yan, L. (2021). Towards DDoS detection mechanisms in software-defined networking. Journal of Network and Computer Applications, 190, article number 103156. doi: 10.1016/j.jnca.2021.103156.
Ferrag, M.A., Maglaras, L., & Janicke, H. (2020). Deep learning for cyber security intrusion detection: Approaches, datasets, and comparative study. Journal of Information Security and Applications, 50, article number 102419. doi: 10.1016/j.jisa.2019.102419 .
Gnatyuk, V., & Gorbachov, I. (2024). Models for improving service quality in IP telephony systems. Science-Based Technologies, 64(4), 456-464. doi: 10.18372/2310-5461.63.19755.
Gnatyuk, V., & Gorbachov, I. (2025). Adaptive resource management in IP telephony using AI to improve QoS. Herald of Khmelnytskyi National University. Technical Sciences, 349(2), 115-121. doi: 10.31891/2307-5732-2025 349-16.
Habib, B., & Khurshid, F. (2024). Time-based DDoS attack detection through hybrid LSTM-CNN model architectures: An investigation of many-to-one and many-to-many approaches. Concurrency and Computation: Practice and Experience, 36(9), article number e7996. doi: 10.1002/cpe.7996.
Hekmati, A., Jethwa, N., Grippo, E., & Krishnamachari, B. (2023). Correlation-aware neural networks for DDoS attack detection in IoT systems. ArXiv. doi: 10.48550/arXiv.2302.07982.
Hussain, M., Khan, M.A., & Ali, S. (2024). Enhanced DDoS detection using advanced machine learning and deep learning techniques. Computers, Materials & Continua, 81(2), 123-145. doi: 10.32604/cmc.2024.057185.
Ilin, D., & Starinskyi, I. (2023). Mathematical model of an intrusion detection system using a neural network based on autoencoders. Modern Information Technologies in the Sphere of Security and Defence, 47(2), 113-118. doi: 10.33099/2311-7249/2023-47-2-113-118 .
Kebede, S.D., Tiwari, B., Tiwari, V., & Chandravanshi, K. (2022). Predictive machine learning-based integrated approach for DDoS detection and prevention. Multimedia Tools and Applications, 81(3), 4185-4211. doi: 10.1007/ s11042-021-11740-z .
Khan, Z.A., & Namin, A.S. (2022). A survey of DDOS attack detection techniques for IoT systems using blockchain technology. Electronics, 11(23), article number 3892. doi: 10.3390/electronics11233892 .
Mittal, M., Kumar, K., & Behal, S. (2023). Deep learning approaches for detecting DDoS attacks: A systematic review. Soft Computing, 27(18), 13039-13075. doi: 10.1007/s00500-021-06608-1.
Najafimehr, M., Zarifzadeh, S., & Mostafavi, S. (2022). A hybrid machine learning approach for detecting unprecedented DDoS attacks. Journal of Supercomputing, 78(6), 8106-8136. doi: 10.1007/s11227-021-04253-x .
Nazih, W., Hifny, Y., Elkilani, W.S., Dhahri, H., & Abdelkader, T. (2020). Countering DDoS attacks in SIP based VoIP networks using recurrent neural networks. Sensors, 20(20), article number 5875. doi: 10.3390/s20205875.
Park, S., Cho, B., Kim, D., & You, I. (2022). Machine learning based signaling ddos detection system for 5G stand alone core network. Applied Sciences, 12(23), 12456. doi: 10.3390/app122312456
Pidpalyi, O., & Romanov, O. (2025). Integration of Zero Trust and Blockchain in SDN networks: An overview of threats and methods of their elimination. Information Technologies and Computer Engineering, 22(1), 55-68. doi: 10.63341/vitce/1.2025.55.
Savchenko, V.A., & Stepanchenko, B.S. (2024). Development of a concept for predicting the start time of a DDoS attack based on the analysis of evolutionary equation dynamics. Telecommunications and Information Technologies, 1, 22-44. doi: 10.31673/2412-4338.2024.012644.
Suvra, D.K. (2025). An efficient real-time DDoS detection model using machine learning algorithms. ArXiv. doi: 10.48550/arXiv.2501.14311.
Xu, Z. (2025). Deep learning based DDoS attack detection. ITM Web of Conferences, 70 article number 03005. doi: 10.1051/itmconf/20257003005.
Zhou, Q., Li, R., Xu, L., Nallanathan, A., Yang, J., & Fu, A. (2024). Towards interpretable machine-learning-based DDoS detection. SN Computer Science, 5, article number 115. doi: 10.1007/s42979-023-02383-y.

Suggested citation

Gnatyuk, V., & Gorbachov, I. (2025). A method for detecting DDoS attacks in VoIP systems based on machine learning. Bulletin of Cherkasy State Technological University, 30(3), 37-46. https://doi.org/10.62660/bcstu/3.2025.37

[1] Chornobuk, M., Dubrovin, V., & Deineha, L. (2023). Cybersecurity: Research on methods for detecting DdoS attacks. Computer Systems and Information Technologies, 4, 6-9. doi: 10.31891/csit-2023-4-1 .

[2] Cui, Y., Qian, Q., Guo, C., Shen, G., Tian, Y., Xing, H., & Yan, L. (2021). Towards DDoS detection mechanisms in software-defined networking. Journal of Network and Computer Applications, 190, article number 103156. doi: 10.1016/j.jnca.2021.103156.

[3] Ferrag, M.A., Maglaras, L., & Janicke, H. (2020). Deep learning for cyber security intrusion detection: Approaches, datasets, and comparative study. Journal of Information Security and Applications, 50, article number 102419. doi: 10.1016/j.jisa.2019.102419 .

[4] Gnatyuk, V., & Gorbachov, I. (2024). Models for improving service quality in IP telephony systems. Science-Based Technologies, 64(4), 456-464. doi: 10.18372/2310-5461.63.19755.

[5] Gnatyuk, V., & Gorbachov, I. (2025). Adaptive resource management in IP telephony using AI to improve QoS. Herald of Khmelnytskyi National University. Technical Sciences, 349(2), 115-121. doi: 10.31891/2307-5732-2025 349-16.

[6] Habib, B., & Khurshid, F. (2024). Time-based DDoS attack detection through hybrid LSTM-CNN model architectures: An investigation of many-to-one and many-to-many approaches. Concurrency and Computation: Practice and Experience, 36(9), article number e7996. doi: 10.1002/cpe.7996.

[7] Hekmati, A., Jethwa, N., Grippo, E., & Krishnamachari, B. (2023). Correlation-aware neural networks for DDoS attack detection in IoT systems. ArXiv. doi: 10.48550/arXiv.2302.07982.

[8] Hussain, M., Khan, M.A., & Ali, S. (2024). Enhanced DDoS detection using advanced machine learning and deep learning techniques. Computers, Materials & Continua, 81(2), 123-145. doi: 10.32604/cmc.2024.057185.

[9] Ilin, D., & Starinskyi, I. (2023). Mathematical model of an intrusion detection system using a neural network based on autoencoders. Modern Information Technologies in the Sphere of Security and Defence, 47(2), 113-118. doi: 10.33099/2311-7249/2023-47-2-113-118 .

[10] Kebede, S.D., Tiwari, B., Tiwari, V., & Chandravanshi, K. (2022). Predictive machine learning-based integrated approach for DDoS detection and prevention. Multimedia Tools and Applications, 81(3), 4185-4211. doi: 10.1007/ s11042-021-11740-z .

[11] Khan, Z.A., & Namin, A.S. (2022). A survey of DDOS attack detection techniques for IoT systems using blockchain technology. Electronics, 11(23), article number 3892. doi: 10.3390/electronics11233892 .

[12] Mittal, M., Kumar, K., & Behal, S. (2023). Deep learning approaches for detecting DDoS attacks: A systematic review. Soft Computing, 27(18), 13039-13075. doi: 10.1007/s00500-021-06608-1.

[13] Najafimehr, M., Zarifzadeh, S., & Mostafavi, S. (2022). A hybrid machine learning approach for detecting unprecedented DDoS attacks. Journal of Supercomputing, 78(6), 8106-8136. doi: 10.1007/s11227-021-04253-x .

[14] Nazih, W., Hifny, Y., Elkilani, W.S., Dhahri, H., & Abdelkader, T. (2020). Countering DDoS attacks in SIP based VoIP networks using recurrent neural networks. Sensors, 20(20), article number 5875. doi: 10.3390/s20205875.

[15] Park, S., Cho, B., Kim, D., & You, I. (2022). Machine learning based signaling ddos detection system for 5G stand alone core network. Applied Sciences, 12(23), 12456. doi: 10.3390/app122312456

[16] Pidpalyi, O., & Romanov, O. (2025). Integration of Zero Trust and Blockchain in SDN networks: An overview of threats and methods of their elimination. Information Technologies and Computer Engineering, 22(1), 55-68. doi: 10.63341/vitce/1.2025.55.

[17] Savchenko, V.A., & Stepanchenko, B.S. (2024). Development of a concept for predicting the start time of a DDoS attack based on the analysis of evolutionary equation dynamics. Telecommunications and Information Technologies, 1, 22-44. doi: 10.31673/2412-4338.2024.012644.

[18] Suvra, D.K. (2025). An efficient real-time DDoS detection model using machine learning algorithms. ArXiv. doi: 10.48550/arXiv.2501.14311.

[19] Xu, Z. (2025). Deep learning based DDoS attack detection. ITM Web of Conferences, 70 article number 03005. doi: 10.1051/itmconf/20257003005.

[20] Zhou, Q., Li, R., Xu, L., Nallanathan, A., Yang, J., & Fu, A. (2024). Towards interpretable machine-learning-based DDoS detection. SN Computer Science, 5, article number 115. doi: 10.1007/s42979-023-02383-y.