Integration of artificial intelligence technologies in data engineering: Challenges and prospects in the modern information environment
Abstract
The integration of artificial intelligence technologies into data engineering gained significant relevancy in the context of constantly growing volumes and complexity of data, which requires innovative approaches to processing and analysis. The goal of the present study is to conduct a deep analysis of the implementation of artificial intelligence into data engineering with a focus on the challenges occurring and perspectives of this process. Research methods, such as analysis methods, comparison, systematisation, and systemic approach, were used for an objective study of this phenomenon and revealing key aspects of this topic. Analysis revealed key challenges, that include variety and instability of data, the importance of standardisation as well as ensuring security of big data amounts. The importance of ethical aspects is underlined, and perspectives on automation of analytical processes and improving prognostic analysis were also determined. According to the results, employment of common standards improves the consistency of approaches, whereas improved algorithms accelerate the processing of big data amounts. Employment of such technology as Apache Hadoop and Spark for processing big data amounts and step-by-step introduction of artificial intelligence is also useful. Increased decision explication also improves their understanding, simplifying interaction between experts and interested parties, and simultaneously creating conditions for effective implementation and employment of integrated artificial intelligence systems in data engineering. The compilation of ethical standards and legal mechanisms creates an opportunity for responsible and balanced employment of these technologies, ensuring trust and ethical compliance in the process of their implementation into various spheres of human activity. These results determine perspectives for the development of this sphere and highlight its importance in a modern informationbased society. Integration of artificial intelligence into data engineering expands capabilities of automating analytical processes, ensuring accurate predictions, and reducing manual labour expenses, creating opportunities for effective management and reasoned decision-making in the data processing sphere
Keywords
intelligent systems; optimisation; security of use; automation of analytical processes; machine learning
References
[1] Ayala-Pazmiño, M. (2023). Artificial intelligence in education: Exploring the potential benefits and risks. 593 Digital Publisher CEIT, 8(3), 892-899. doi: 10.33386/593dp.2023.3.1827.
[2] Baird, A., & Schuller, B. (2020). Considerations for a more ethical approach to data in AI: On data representation and infrastructure. Frontiers in Big Data, 3, article number 25. doi: 10.3389/fdata.2020.00025.
[3] Bharati, S., & Podder, P. (2022). Machine and deep learning for IoT security and privacy: Applications, challenges, and future directions. Security and Communication Networks, 2022, article number 8951961. doi: 10.1155/2022/8951961.
[4] Danchenko, U.M. (2023). Relevance of the study of artificial intelligence recipients of higher military education. In Materials of the all-Ukrainian science-pedagogical professional development “Technologies for the benevolent use of artificial intelligence in the fields of education and science” (pp. 83-85). Odesa: Helvetica.
[5] Ebid, A.M. (2021). 35 Years of (AI) in geotechnical engineering: State of the art. Geotechnical and Geological Engineering, 39, 637-690. doi: 10.1007/s10706-020-01536-7.
[6] Fui-Hoon Nah, F., Zheng, R., Cai, J., Siau, K., & Chen, L. (2023). Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration. Journal of Information Technology Case and Application Research, 25(3), 277-304. doi: 10.1080/15228053.2023.2233814.
[7] Gill, S.S., Xu, M., Ottaviani, C., Patros, P., Bahsoon, R., Shaghaghi, A., Golec, M.,... Uhlig, S. (2022). AI for next generation computing: Emerging trends and future directions. Internet of Things, 19, article number 100514. doi: 10.1016/j.iot.2022.100514.
[8] Gröger, C. (2021). There is no AI without data. Communications of the ACM, 64(11), 98-108. doi: 10.1145/3448247.
[9] Hbur, Z. (2022). Use of artificial intelligence in information security in Ukraine. Public Administration: Improvement and Development, 1. doi: 10.32702/2307-2156-2022.1.2.
[10] Jöhnk, J., Weißert, M., & Wyrtki, K. (2021). Ready or not, AI comes – an interview study of organizational AI readiness factors. Business & Information Systems Engineering, 63, 5-20. doi: 10.1007/s12599-020-00676-7.
[11] John-Mathews, J.M., Cardon, D. & Balagué, C. (2022). From reality to world. A critical perspective on AI fairness. Journal of Business Ethics, 178, 945-959. doi: 10.1007/s10551-022-05055-8.
[12] Khaustova, V., Reshetnyak, O., & Khaustov, M. (2022). Promising areas of IT development in the world. The Problems of Economy, 51(1), 3-19. doi: 10.32983/2222-0712-2022-1-3-19.
[13] Martínez-Fernández, S., Bogner, J., Franch, X., Oriol, M., Siebert, J., Trendowicz, A., Vollmer, A.M., & Wagner, S. (2022). Software engineering for AI-based systems: A survey. ACM Transactions on Software Engineering and Methodology, 31(2), article number 37e. doi: 10.1145/3487043.
[14] Mohan, Y., & Singh, A.P. (2022). Challenges and opportunities in Big data: A review. International Journal of Research in Electronics and Computer Engineering, 10(4), 36-46.
[15] Patel, A.R., Ramaiya, K.K., Bhatia, C.V., Shah, H.N., & Bhavsar, S.N. (2021). Artificial intelligence: Prospect in mechanical engineering field – A review. In K. Kotecha, V. Piuri, H. Shah & R. Patel (Eds.), Data science and intelligent applications. Lecture notes on data engineering and communications technologies (pp. 267-282). Singapore: Springer. doi: 10.1007/978-981-15-4474-3_31.
[16] Sarker, I.H. (2022). AI-based modeling: Techniques, applications and research issues towards automation, intelligent and smart systems. SN Computer Science, 3, article number 158. doi: 10.1007/s42979-022-01043-x.
[17] Serey, J., Quezada, L., Alfaro, M., Fuertes, G., Vargas, E., Ternero, R., Sabattin, J., Durán, C., & Lillo, S. (2021). Artificial intelligence methodologies for data management. Symmetry, 13(11), article number 2040. doi: 10.3390/ sym13112040.
[18] Shevchenko, A.I. (Ed.). (2023). Strategy for artificial intelligence development in Ukraine. Kyiv: Nauka i Osvita. doi: 10.15407/development_strategy_2023.
[19] Shmatkovska, T., Korobchuk, T., & Borysiuk, O. (2023). Modern information and communication technologies in the system of accounting and analytics for the modeling of business processes. Economy and Society, 53. doi: 10.32782/2524-0072/2023-53-68.
[20] Talib, M.A., Majzoub, S., Nasir, Q., & Jamal, D. (2021). A systematic literature review on hardware implementation of artificial intelligence algorithms. The Journal of Supercomputing, 77, 1897-1938. doi: 10.1007/s11227-02003325-8.
[21] Zachek, O., Dmytryk, Y., & Senyk, V. (2023). The role of artificial intelligence in increasing efficiency law enforcement activities. Scientific Bulletin of the Lviv State University of Internal Affairs, 3, 148-156. doi: 10.32782/2311-8040/2023-3-19
[22] Zhang, C., & Lu, Y. (2021). Study on artificial intelligence: The state of the art and future prospects. Journal of Industrial Information Integration, 23, article number 100224. doi: 10.1016/j.jii.2021.100224.