Integrating Model-Driven Engineering with Machine Learning for Intelligent Systems: Literature Review ^†

Abstract

The rapid development of intelligent systems has created a need for techniques capable of handling complexities and developing in an automatic way. The integration of machine learning in model-driven engineering (MDE) offers several advantages for the development and improvement of complex and intelligent systems. While machine learning (ML) also offers robust techniques and MDE has systematic approaches aimed at code generation and abstraction, in this review, while presenting the principles of MDE and ML, the article also critically explores the integration of ML in MDE. Starting with the fundamental concepts of MDE, then the principles and algorithms of ML, the focus of the discussion is on how machine learning techniques can improve model-driven engineering processes. By presenting the motivations for their combined use in the development of intelligent systems, based on the recent literature, the article describes the challenges and potential future directions, noting that the integration of machine learning into model-driven engineering not only accelerates development but also enhances the adaptability and performance of intelligent and complex systems, making it an increasingly relevant approach to addressing the complexities of modern intelligent systems.

1. Introduction

The emergence of complex and intelligent systems requires the development of methodologies that can cope with both the complexity of system requirements and the speed of technological innovation. MDE and ML have emerged as two essential paradigms to address these challenges. MDE focuses on raising the level of abstraction of software development by using models as primary artifacts, thereby simplifying design, validation, and code generation. In contrast, ML focuses on extracting patterns and knowledge from large datasets, thereby enabling systems to learn and adapt over time. The beneficial possibilities are offered by the joint approach of these paradigms. By integrating machine learning techniques into the work of model-driven engineering, it can lead to improved automation, enhanced predictive capabilities, and the development of systems capable of handling random scenarios. The article provides an in-depth exploration of the theoretical foundations of MDE and ML. It further details how linking the approaches can contribute to the evolution of intelligent systems. The review is structured into three main sections: the first section discusses the principles of MDE, the second section provides an overview of ML, and the third section presents a comprehensive analysis of the integration of MDE and ML. In doing so, this work aims to provide researchers and practitioners with a synthesized understanding of the current state of the art, as well as directions for future research.

2. Model-Driven Engineering

“MDE is a software development approach that emphasizes using models as a primary means for specifying, designing, and implementing software systems. A simple and popular formula used to describe MDE is the following: Models + Transformations = Software” [1,2]. To provide important context, we present the key concepts involved. “A model is an abstract representation of a system that eliminates unnecessary details to increase the users’ understanding of the system under consideration” [1]. A metamodel, in turn, is a specific type of model that defines the structure and rules of a modeling language; it serves as the abstract syntax of that language. Model transformations are specialized programs that convert one model into another or into textual representations (source code is an example), based on predefined transformation rules. Transformations are of two types: model-to-model, and model-to-text, where the input is a model and the output is textual code in languages such as Java.

MDE is a software development approach based on the construction and use of high-level abstract models; these models represent various aspects of a system. Models can be analyzed in an organized manner, validated, and transformed using dedicated tools to produce executable and efficient software. The primary goal of MDE is to improve development productivity and software quality by enabling a more abstract design process, thus reducing the gap between system specification and implementation. MDE is particularly beneficial for large and complex systems, where traditional development methods can become inefficient. It uses modeling languages such as UML or SysML and domain-specific languages (DSLs), as well as supporting tools such as model checkers and transformation engines for M2M and M2T processes [1,2,3].

3. Machine Learning

ML is a subfield of artificial intelligence (AI) that focuses on developing algorithms that can learn from data. ML allows systems to improve their performance over time without being explicitly programmed for each task. ML is categorized into different types of learning. There is unsupervised learning, which identifies patterns in unlabeled data through strategies such as clustering and dimensionality reduction, commonly applied to customer segmentation and document categorization; and there is supervised learning, which is used for classification and prediction tasks, based on labeled data to map inputs to desired outputs. Algorithms such as decision trees, support vector machines, and artificial neural networks fall under this category. Semi-supervised learning combines principles of both supervised and unsupervised approaches, making it useful when labeled data is sparse. Reinforcement learning works on a reward-based system, hence it is usable to improve and develop systems and applications, such as robotics, autonomous driving, and AI games [4].

4. Advantages of Integrating ML into MDE

Integrating machine learning into model-driven engineering (see Figure 1) offers serval advantages. For example, enhanced automation, whereby developers automate code generation in an efficient and more convenient way, based on the use of machine learning algorithms to predict the greatest model transformation paths. Improved adaptability is mandatory to adapt intelligent systems to changing and developing contexts as well. Incorporating machine learning into model-driven engineering workflows enables systems to learn from operational data and adjust models accordingly. Also, bridging the abstraction gap is an essential benefit for integrating ML into MDE. ML requires detailed data and algorithms, but MDE often operates at high levels of abstraction. Integration enables a bi-directional flow where high-level models guide the use and configuration of ML algorithms, and insight from ML inform model refinement and increase robustness.

4.1. Approaches to Integration MDE and ML

Integration of ML into modeling, in particular modeling languages, should be extended to enable seamless incorporation of both model-learned attributes and “default” ones. Modeling language must be specific in a fine-grained manner about what must be learned, similar to how machine learning algorithms learn, and also specify why something must be learned. Learning units should be independently computable and updateable for it to be appropriate for learning. So, there is a meta–meta model to define and specify this weaving because a meta model defines what can be represented in meta models that conform to it, or what notions may be expressed in a real meta model. This makes it possible for domain modes to convey learning issues. This allows us to build a concrete modeling language that includes the constructs required to integrate machine learning into domain modeling [5]. In this approach, there are micro learning units that are usable for small fine-grained learning units and the micro learning units for decomposed and structured complex learning tasks with reusable and computable elements. Modeling language: The modeling language for domain data, and its structure and associated learning unit. To avoid ambiguities in usable language to specify this language, the following definitions for the latest developments in meta-modeling language served as inspiration for the language. For example: UML, SysML [5,6].

4.2. Addressing ML Development Challenges with Model-Driven Engineering (MDE)

It might be difficult to develop and manage systems that use ML models and components. Immature requirement definition, continuously changing data, a lack of domain knowledge in machine learning, integration with conventional software, responsible usage of machine learning, and the deployment and upkeep of ML models are some of the components of this complexity. These complexities present a number of challenges. Nils Baumann et al. [7] explain this well; for example, regarding the difficulty of managing evolving datasets, machine learning engineers must manually combine new and old datasets and retrain the entire machine learning model. According to Benjamin Jahi et al. [8], describing the dataset and neural network well to meet customer expectations is difficult; according to Benjamin et al., creating a highly efficient machine learning pipeline is a very difficult and complex job too; data scientists need to be skilled and experienced to sort through a variety of data preprocessing and machine learning models and choose the best one. Kaan Koseler et al. [9] discuss the challenges developers face when trying to apply machine learning techniques with large amounts of data; developers must be familiar with the problem space, domain, and concepts of machine learning. To successfully and efficiently manage and control these problems, solutions are needed. Software models are used to guide the development and administration of ML components, demonstrating the synergy between MDE and ML development. This is not the same as AI or machine learning for MDE (AI4MDE), which helps users with modeling and related tasks using highly intelligent agents and recommendation systems. There are several very potential benefits for developers using MDE for AI-based systems (MDE4ML), including less complexity, time, and effort. Through the abstraction and automation of MDE, AI can also be used by software engineers, subject matter experts, and AI novices. In addition, MDE can improve the quality of the AI-based system by making it easier to maintain, scale, reuse, and interoperate.

4.3. Integrating Machine Learning into Model-Driven Engineering: Approaches, Strategies, and Benefits

A group of researchers addressed the work on the integration of ML in model-driven engineering through their studies and presented various examples in an effective and smooth manner. Naveed et al. [1] did a systematic literature review (SLR) on the practice and use of MDE, improving and developing machine learning components. The research of these researchers highlights different structures and strategies to adapt model transformations to meet the important and specific requirements of machine learning, specifically regarding data flow management, training parameters, and evolving nature of machine learning models over time, by applying the principles of model-driven engineering. So, the research explains how automated transformations can streamline the machine learning development process and ensure better alignment within high-level designs and their corresponding implementations. [1]. On the other hand, Rädler et al. [10] has also explored the use of SysML to improve and generate ML code, presenting high-level system models on how they can be directly translated into executable and practical ML code. The approach by Rädler et al. [10] significantly facilitates the transition from design to implementation, ensuring that the generated code remains compliant with the initial system specifications. By applying SysML, complex ML workflows are defined by developers in a structured manner, thereby enhancing both the organization and efficiency of ML software development. Thus, this approach optimizes manual coding efforts and minimizes inconsistencies between conceptual models and final implementations [10]. On the other hand, Hartmann et al. [5] present a particularly representative example of the seamless integration of machine learning into domain modeling. To create models that are inherently aware of ML components, domain-specific languages (DSLs) are used in this approach. These models are enhanced to directly integrate machine learning algorithms, creating a direct mapping between model elements and AI features. According to the presented approach, the integration offers several key benefits. First, it improves the traceability and ease of system architecture, thus facilitating the updating and adaptation of models as technology evolves. It also significantly minimizes development time by automating parts of the process of generating and using machine learning models. Furthermore, the use of DSLs allows every aspect of the model to align closely with the underlying machine learning mechanisms, thus helping to improve consistency and interpretability during the system lifecycle [5].

4.4. Analysis of Scientific Studies in MDE

In this section we will present our analysis of some scientific studies of MDE based on the following criteria: languages, quality assessment, application domains, and methodologies. The following table (

Table 1. Classification of papers.

Criteria	Classification of Papers by ML with MDE	Classification of Papers by Model Transformation Language	Classification of Papers by Application Domain (Intelligent Systems)	Classification of Papers by Modeling Language for Developing IoT Applications
Papers	Naveed et al. (2024) [1] Kelly et al. (2024) [3] Lee et al. (2024) [4] Brambilla et al.(2024) [11] Ciccozzi et al. (2024) [12] López et al. (2024) [13]	XTend Jahić et al. (2023) [8] Ries et al. (2021) [14] García-Díaz et al. (2015) [15] Epsilon Generation Language (EGL) Koseler et al. (2019) [9] Yohannis et al. (2022) [16] Kourouklidis et al. (2021) [17] Al-Azzoni et al. (2020) [18] MontiAnna/MontiArc generators Baumann et al. (2022) [7] Gatto et al. (2019) [19] Kusmenko et al. (2019) [20] Acceleo Espinosa et al. (2029) [21] Tabbiche et al. (2023) [22] Safdar et al. (2022) [23] Atlas Transformation language (ATL) Mili et al. (2012) [24] Santos et al. (2018) [25] Krstić et al. (2022) [26] TouchCore Shi et al. (2022) [27] Apache Velocity Hartmann et al. (2017) [5] OPC UA code generator Shin et al. (2020) [28]	Healthcare Brambilla et al. (2017) [5] Mehrabi et al. (2022) [29] Meliá et al. (2021) [30] Kotronis et al. (2018) [31] Morin et al. (2016) [32] Mezghani et al. (2017) [33] Veňckauskas et al. (2016) [34] Agricultural City Energy Manufacturing Barriga et al. (2022) [35,36,37] Nepomuceno et al. (2020) [38] Ziaei et al. (2020) [39] Building Berrouyne et al. (2022) [40] Barriga et al. (2022) [36,37] Berrouyne et al. (2020) [41] Kirchhof et al. (2022) [42] Alulema et al. (2021) [43] Environment Karaduman et al. (2020) [44] Asici et al. (2019) [45] Durmaz et al. (2017) [46] Transport Anwer et al. (2020) [47] Berrouyne et al. (2022) [40]	DSLs Veňckauskas et al. (2016) [34] Erazo-Garzón et al. (2022) [48] Ihirwe et al. (2021) [49] Karaduman et al. (2021) [37,50] UML Parri et al. (2021) [51] Karaduman et al. (2021) [50,52] Plazas et al. (2020) [53] Jahed et al. (2019) [54] Moreira et al. (2019) [55] BPMN Moreira et al. (2019) [55] Sosa-Reyna et al. (2018) [56] SysML Costa et al. (2016) [57]

) presents a classification of the articles that are treated in this study. Articles are classified among the classification of ML and MDE, i.e., papers. Papers are classified by model transformation language: Xtend, Epsilon Generation Language (EGL), MontiAnna/MontiArc generators, Acceleo, Atlas Transformation language (ATL), TouchCore, Apache Velocity, Langium, OPC UA code generator P45. Then, classification of papers is performed by application domain (intelligent systems): Healthcare, Agricultural and City and Energy, Manufacturing, Building, Environment, Transport. Finally, classification of papers is performed by modeling language for developing IoT applications: DSLs, UML, BPMN. The papers discuss the main challenges and issues of MDE. Others present how to use this approach to model systems. An article discusses the application of model transformations in the process of intelligent systems engineering.

5. Conclusions and Future Work

In this paper, we have presented the main roles of model-driven engineering and machine learning in the development of intelligent systems. MDE is a systematic approach to system development with an emphasis on abstraction, automation, and model transformations, and ML provides powerful methods to learn from data and adapt to changing environments. Therefore, developers can achieve a higher degree of automation, better adaptability, and increased robustness of the system, based on the integration of these two techniques. In our future work, we plan to further explore the integration of MDE with ML techniques. In particular, we aim to develop a framework that facilitates the automated generation and refinement of models through learning from existing data, and to develop an innovative intelligent system in a specific field using our MDE approach and ML techniques.

6. Discussion and Research Perspectives

This study offered a clear and structured overview of the different current engineering approaches led by models (MDE) for smart systems. If it highlights the important advances already made, it also reveals several tracks that are still little explored. For example, we especially note the lack of link between the MDE and the explanatory automatic learning techniques (explaining ML). In a world where user transparency and confidence have become essential, integrating this explainable dimension in MDE processes seems to be a promising path. Future research could thus be interested in how to combine performance and understanding of systems by their users.

Another point that deserves more attention is that of automatic refinement of models. Despite progress in synthesis and transformation of models, few studies have really explored how to adapt the models automatically and continuously in the face of changes in needs or returns in real time. This area represents a great opportunity to develop MDE frameworks capable of dynamically adapting to evolutionary contexts.

In summary, it is important that future research does not only solve current technical problems, but that they also incorporate interdisciplinary approaches. By bringing the MDE closer to domains such as artificial intelligence or software engineering, we can expand horizons and significantly advance the design of smart systems of tomorrow.