Extending the OSLC Standard for ECA-Based Automation

Abstract

Workflow automation is taking over software development systems, helping businesses increase efficiency, accelerate production, and adapt quickly to market changes. Combined with agile principles, it has given birth to the DevOps paradigm. However, practitioners often face an important issue known as vendor lock-in caused by the cost of tool replacement or migration to different platforms. This issue could be addressed by standardising service interfaces to facilitate their integration. Linked Data is an attractive choice for implementing such a standard without sacrificing versatility. Following this approach, the Open Services for Lifecycle Collaboration (OSLC) proposal aims to build an environment where services can interoperate using standard models. Therefore, this article proposes an extension of the existing OSLC specification, based on the Event-Condition-Action (ECA) model, for event-based interoperable automation. This extension enables a new path in the field of semantic automation for OSLC services, which allows the self-interaction of services among them and with human users. The article presents the key concepts of the proposed model and exemplifies its application in an automation scenario with two real-life services. The validation of the proposal has been carried out using established ontology evaluation methods, such as coverage and similarity metrics and competency questions.

1. Introduction

In recent years, there has been explosive growth of DevOps as a development method in the software industry. It helps companies deliver their software products to customers efficiently and effectively [1,2]. The critical aspects of the DevOps methodology are the emphasis on the union between software developers and IT operators. That collaboration relies on software delivery automation, accelerating the process and increasing the quality and reliability of the product, both critical aspects in an ever-changing business environment. Hence, automation is crucial in DevOps [3,4]. That is because it dramatically reduces the number of repetitive tasks and the risks of human error. Additionally, automation gives developers and operators more time to perform other different tasks, increasing the efficiency of the software development process.

Currently, there is a multitude of automation tools available for DevOps practitioners for configuration or deployment, such as Ansible [5], Puppet [6], Chef [7], Jenkinks [8] or Terraform [9]. However, only a few of them work as Task Automation Server (TAS) [10], which is the most interesting part of automating the DevOps process. These TAS frameworks are often configured with a popular model for automation rules, the ECA model [11]. For general web users, this kind of automation has several successful platforms for a broad audience, such as IFTTT [12] or Zapier [13]; but, in the context of software development, tools such as StackStorm [14] and Clarive [15] apply the same ideas to automate production and delivery tasks.

However, given that each service that streamlines the software development process is a potential source of events and offers a possible set of actions to execute, it makes the number of tools that could be integrated into the automated streamlining unmanageable [16]. Therefore, the number of integrated tools is limited to the subset of TAS developers decide to include in their ecosystems. Even when the open source community supports these types of tools, there are still two critical challenges associated with TAS frameworks in DevOps: (i) the potentially high cost of integrating new services [17] and (ii) the lack of flexibility when automating processes [18].

A potential solution to the integration problem (i) found in the literature [19] would be to standardise the interfaces between the different tools, making the migration process fast and seamless. However, this is a complex task. Every tool has its particular model and architecture, so it would be difficult for any possible standard to adjust appropriately to all of them. In this context of service interoperability, solutions based on semantic web technologies have been considered before [20]. Following the principles of Linked Data, projects are already attempting to standardize the interfaces between lifecycle tools using Linked Data. In particular, the OSLC project [21] stands out for its innovative approach to tool interoperability.

In the automation problem (ii) context, standardising how the services involved are modelled could also benefit flexibility improvements for automation solutions [22]. Again, the challenge is to design a sufficiently flexible model so that new services can be directly integrated without requiring them to adjust to a restrictive model. In this aspect, there is a specific domain in OSLC for automation concepts [23] oriented to configuration and deployment tasks. However, it has some limitations when modelling a task automation service based on the event-condition-action model. In particular, it needs some necessary concepts to represent events, actions, or rules; and crucial relations among them, which we face in the present article.

Therefore, the main contribution of this paper is to propose an extension for the OSLC Automation domain [23] to support ECA-based automation. This extension bridges the gap between the OSLC approach and the ECA model. This extension enables a new path in the field of semantic automation for OSLC services, which allows the self-interaction of services among them and with human users. The article presents the key concepts of the proposed model and exemplifies its application in an automation scenario with two real-life services.

The remainder of this paper is structured as follows. Section 2 presents related works that have laid the basis for the contribution. Section 3 proposes the OSLC-ECA model, including examples of its applications in a proof-of-concept prototype. Section 4 evaluates the proposed semantic model, comparing it to the current OSLC Automation specification. Finally, the conclusions and possible future work are described in Section 5.

2. Related Work

This section presents related work in the field of knowledge related to this paper. First, the most popular commercial tools for implementing ECA automation in DevOps are analysed in Section 2.1. Later, we review projects in which OSLC was used to integrate different software tools, paying particular attention to those that involve automation tools in Section 2.2.

2.1. Event-Condition-Action Automation Tools for DevOps

Software companies use a vast set of commercial tools to automate their processes. This section compares some of the most popular alternatives for automation tools in the DevOps environment. For our study, we chose four popular services: StackStorm [14], Clarive [15], GitHub Actions [24], and GitLab CI/CD [25].

First, StackStorm and Clarive are business-oriented tools that conform to the ECA model. StackStorm has become noticeable both in the literature [26] and in the industry [27]. Clarive has also experienced a recent increase in popularity [28]. In addition to implementing other extra features, their most basic concepts are drawn from the ECA model. For this reason, we chose to study them as references for automation tools in the market. Second, GitHub and GitLab are the most popular platforms for developers to store and manage their code. GitHub has over 90 million active users [29], and GitLab has an estimated 30 million registered users, 1 million of them actively using its DevOps platform [30]. Their integrated automation solutions, i.e., GitHub Actions and GitLab CI/CD, respectively, are also widely adopted and must be considered for our study. Although their models do not strictly implement the ECA model, they can be used to represent most of them at a high level. They must be considered in the following comparison due to their huge popularity in the industry. The comparison of how these four tools fit the ECA model is shown in

Table 1. Comparison of nomenclature for automation providing tools.

Concept	StackStorm	Clarive	GitHub Actions	GitLab CI/CD
Event	Trigger	Event	Event	Trigger
Action	Action	Ops	Action	Job
Rule	Rule	Rule	Workflow	Pipeline

.

Although the tools chosen implement many features beyond the creation of automation rules, we summarised some of them to show the technical differences between these tools and the need for more standardisation that we aim to address in this work.

Table 2. Comparison of features for automation providing tools.

Feature	Stack Storm	Clarive	GitHub Actions	GitLab CI/CD
Rules with more than one condition	No	No	Yes	Yes
Rules with more than one action	Yes	Yes	Yes	Yes
Rule sharing	Yes	Yes	Yes	Yes
Custom actions	Yes	Yes	Yes	Yes
Events generated outside own platform	Yes	Yes	No	No
Rules API	No	Yes	Yes	Yes
Rule chaining	Yes	Yes	Yes	Yes

summarises the main relevant features of automation in a DevOps environment. However, as can be observed, they all suffer some deficiencies compared to the others. The features were extracted by performing an exhaustive search of the documentation for each tool. This feature extraction process is detailed in Section 4.1, where the proposed model is evaluated.

2.2. Automation with OSLC Standard

Currently, the OSLC project [31] aims to develop a standard interface for software development tools that applies semantic technologies provided by World Wide Web Consortium (W3C). It consists of open specifications for tool integration using the Linked Data initiative. The project is based on a core specification [21] that every OSLC-compliant tool must follow and a series of domain-specific protocols that are more suited to the particular needs of some groups of tools.

OSLC offers a unique combination of standardisation and flexibility. Modelling automation tools with OSLC could solve many common vendor lock-in problems [32]. In [33], the authors conducted an interesting systematic mapping study to know the state of the art for applying the OSLC standard for model integration to enable automation between services. They included 59 primary studies and applied a numeric scale to answer research questions and competency assessment criteria to compare the studies among them. In conclusion of the study, the authors claim that the advantages of OSLC are facilitating tool replacement in the toolchain and modifying specifications for tool integration in automated activities. Different projects involving OSLC have been studied to learn more about previous attempts to apply the standard to the automation domain, as mentioned in the following paragraphs.

Beyond the field of software engineering, architectures that integrate IoT devices using OSLC have been proposed [34]. Due to the heterogeneity of these IoT devices, ensuring their interoperability becomes a challenge that the authors tackle using Linked Data and a standard communication protocol. The result of the proposed architecture is a unified model with all data modelled using the OSLC standard through different OSLC Adapters, i.e., modules that translate and transform data from non-semantic models to OSLC-compliant models. However, the authors mention the need for an specific ontology for every service and vendor to automate IoT system development as a limitation.

Another interesting approach is found in the CRYSTAL-ARTEMIS project [35,36], a previous work that implemented an interoperability platform based on OSLC. Although OSLC is used as the foundation for the project, its models go far beyond the proposed OSLC standards. This work aims to define a model that could serve as a potential new specification and is strongly industry-oriented, pushing the Interoperability Specification towards standardisation. However, the proposed semantic models are not explicitly designed for ECA automation, which is the main objective of this work.

Based on this previous work, another OSLC-based architecture integrates testing and requirement tools [37] in the European Rail Traffic Management System/European Train Control System context. In this case, the OSLC specification is used in domains such as Requirement Management, Architecture Management, or Quality Management, defining OSLC Adapters to translate data to an OSLC-compliant format. Furthermore, the OSLC Automation domain is used in this work to integrate code analysis tools with the popular Eclipse IDE [38].

In addition to the work mentioned above, in which interoperability is emphasised primarily to enable integration and automation but not explicitly focused on an event-condition-action model, the ECA model has been applied to design automation tools in different domains. For example, in the context of IoT, ECA rules have been used to simplify the implementation of building automation applications [39] proposing an interesting architecture that compares with previous works; or an ECA-based interoperability framework for intelligent building [40]. Nevertheless, we find an architecture proposal that includes both to achieve the ECA automation and OSLC standard in the same work [41]. It uses the ECA model to integrate a toolchain for IoT and uses Tracked Resource Set (TRS) [42]. This OSLC-based protocol tracks changes in OSLC resources and entities to automate different tasks. They affirm that this approach would also be very beneficial in Big Data environments because “using microservices and Linked Data to integrate systems within the toolchain eliminates the need for multiple transformations” [41], achieving better velocity and variety than previous techniques. They propose a case study of a robotic warehouse optimised for logistics automation. To communicate changes in these tools, they apply the MQTT protocol [43] to keep everything synchronised. This work is promising because it integrates various tools and devices using semantic technologies and creates a centralised communication network between these tools, allowing them to automate their behaviour. However, there is a need for a semantic model yet to be explicitly designed for ECA, as TRS is a change tracking protocol, not created with the core idea of ECA-based automation in particular. Therefore, the following section presents the proposed OSLC-ECA model as the main contribution of this paper.

3. The Proposed OSLC Extension for ECA-Based Automation

This section presents the article’s main contribution, the semantic model that extends OSLC specification to support ECA-based automation. The most important concepts of the proposed model are presented in the following sections, but the complete specification of this proposal is available online [44]. Section 3.1 presents the iterative methodology followed to generate the proposed model. Section 3.2 briefly summarises the external ontologies used in the proposal. Section 3.3 details the main classes and properties included in the proposed OSLC-ECA model. Finally, Section 3.4 shows some examples of the use of the model in a case study, and the prototype’s architecture is briefly explained in Section 3.5.

3.1. Design Methodology

The proposed model has been generated following an iterative development methodology consisting of three phases. First, (i) we analyse the most relevant automation tools in the field (identified in Section 2) and extract the features and functionalities existing in them as concepts and properties. Second, (ii) we define a model that formally describes these concepts following OSLC principles. Finally, the model is evaluated against the various use cases considered (for example, the actions described on the GitHub Actions platform).

In each iteration, we include new elements, if needed, that are relevant to the results tested and we redesign others to describe the domain best. So, each iteration refines the model making it more valuable and accurate, including modifying or removing classes and properties.

3.2. Mappings from External Ontologies in OSLC-ECA

The proposed OSLC-ECA model [44] has been developed to extend the existing OSLC Specification. This section introduces some basic concepts of the complex and broad OSLC project, which are crucial to understanding the contribution presented in the following sections. OSLC [45] project is a family of open specifications for the integration of tools. Its purpose is to integrate product/application lifecycle services regardless of their internal model. In OSLC, different domains, such as change management [46], or requirement management [47], define the specifications of their specific semantic concepts. All these domains are built on top of the OSLC Core Specification [21] to ensure their compatibility. For further details, the complete specification must be referred to [21,23]. The relations between OSLC-ECA and other existing OSLC domains are described in the following paragraphs and shown in Figure 1.

• OSLC Core v3.0 [21]: Following this specification, the data are semantically represented with Resource Description Framework (RDF) and are retrieved via an HTTP API. Therefore, each service must provide a catalog of its oslc:ServiceProviders, which must be understood as “organizing concepts that partition the overall space of artifacts in the tool into smaller containers.” [45]. This definition makes the concept of oslc:ServiceProvider different depending on the type of service that we were handling based on its domain. Potential clients can discover all those providers by discovering all resources and capabilities based on its root resource, the oslc:ServiceProviderCatalog. This resource allows clients to obtain its list of oslc:ServiceProviders responsible for providing the information needed to send any well-formed request to the oslc:Service, allowing clients to interact appropriately. These endpoints can be discovered through the oslc:CreationFactory and oslc:QueryCapabilities services. They may also provide an oslc:ResourceShape that describes the properties a resource from that service is expected to have.
• OSLC Automation v2.1 [23]: Regarding the process automation field, OSLC defines a specific domain called OSLC Automation. This domain aims to model an interface so that an automation provider can interact with other oslc:Services. It defines three different crucial resources that a oslc:ServiceProvider exposes. The first of these resources is the oslc_auto:AutomationPlan which defines a unit of automation available for execution. Next, the oslc_auto:Automation Request resource provides the information required to execute an oslc_auto:AutomationPlan. Finally, the standard defines an oslc_auto:AutomationResult resource to track an oslc_auto:AutomationRequest status.
• LDP [48]: The W3C Linked Data Platform (LDP) supports HTTP operations on web RDF resources to provide a web-based read–write Linked Data architecture. In this case, the concept ldp:Container is used in the same way as that in OSLC Core Specification [21] adapted to the ECA domain: a ldp:Container provides a URL where it can be used to interact with actions of a specific service.
• OWL [49]: In this case, the Web Ontology Language (OWL) ontology allows flexibility in defining the source of events and how the proposed OSLC-ECA model can recover them.
• DC [50]: Dublin Core is used for most of the properties of every class, defined according to DCMI Metadata Terms metadata elements. This is similar to the way it is used in the OSLC Core Specification.

3.3. OSLC-ECA Elements

As mentioned previously, there is a need for critical concepts for ECA-based automation in the current version of the OSLC specification. For clarity, all new concepts (classes and properties) proposed in the article use the oscl_eca prefix.

The model’s core comprises three significant classes: oslc_eca:Event, oslc_eca:Action, and oslc_eca:Rule. The description of a particular scenario may inherit from them, creating new classes specific to the domain using the OSLC specification as detailed below.

3.3.1. Event Modelling

This section details the class oslc_eca:Event, the related class oslc_eca:EventListener, and their most relevant properties, summarised in the diagram shown in Figure 2 and explained in the following paragraphs.

The class oslc_eca:EventListener represents a background process that detects events. Therefore, any agent interested in a specific event will be notified when it occurs. It can be considered as the monitoring of events from a specific source and service, modelled with the oslc_eca:source property that relates an oslc_eca:EventListener to its source, modelled as a generic owl:Thing. That source must be specified for each domain. An oslc:ResourceShape describes the oslc_eca:EventListener class as any other OSLC resource, and an oslc:ServiceProvider provides it.

The process of receiving/retrieving events can be implemented following different strategies, among others: using webhooks, providing a URL to the service to make a POST request with the new events; with a publish-subscriber protocol, such as MQTT, which has been used before in a similar context [41]; or using some messaging service that can deliver the events to their intended receivers, such as the Kafka distributed event streaming platform [51].

Thus, the option used by an oslc_eca:EventListener to receive/retrieve events is defined by the oslc_eca:binding property, similar to [52]. Again, this property is related to the owl:Thing class and must be specified with a specific class for each domain, opening the door to new interaction patterns between the source of events and the event listener.

The class oslc_eca:Event represents a concrete instantaneous fact that is not extended in time. Whenever an event listener detects a new event, it generates an oslc_eca:Events individual as an OSLC resource, which will be described by a oslc:ResourceShape that describes the specific properties of a kind of event.

oslc_eca:Event individuals are related to their source through the oslc_eca:generatedBy property, which relates them to the oslc_eca:EventListener individual that detected the event, and through the oslc_eca:causedBy property, which relates them to the specific OSLC Resource that causes the event.

Finally, there is a relation between these ECA events and the OSLC Automation domain. This is because an oslc_eca:Event could be used to trigger oslc_auto:AutomationPlans. Therefore, there should be a relationship between the oslc_eca:Event and oslc_auto:AutomationRequest classes. An oslc_auto:AutomationRequest could be triggered when an oslc_eca:Event occurs. This relation is modelled with the property oslc_eca:triggeredBy.

3.3.2. Action Modelling

This section details the class oslc_eca:Action, the related class oslc_eca:ActionDispatcher, and their most relevant properties. Figure 3 shows a diagram representing them, followed by their explanation in the following paragraphs.

The class oslc_eca:Action represents an action and provides the instructions to execute it and determine its result. Individuals in this class must have an oslc_auto:State and an oslc_auto:Veredict through the oslc_auto:state and oslc_auto:veredict properties of the OSLC Automation Specification [23]. The oslc_eca:Action is described by a oslc:ResourceShape as any other OSLC resource and is executed on other OSLC resources through the oslc_eca:executedOn property.

Similarly to the oslc_eca:EventListener class, the oslc_eca:ActionDispatcher class is defined to represent the capability of a service or agent to execute specific actions. An individual in this class that executes a concrete oslc_eca:Action is specified by the oslc_eca:executedBy property.

The oslc_eca:execution property signals an endpoint where consumers can send post-requests to execute the action it executes. Actions sent for execution can be queried using another endpoint provided by the oslc_eca:ActionDispatcher through the oslc_eca:queryActions property. These URLs are represented by ldp:Container objects, as in the OSLC Core specification [21].

The standard establishes a relationship between the OSLC Automation [23] and Actions [52] specifications. Specific oslc_eca:Actions can become available (or be automatically executed) when an oslc_auto:AutomationPlan successfully finishes running. The oslc_eca:futureAction property was defined to identify them. In this extended model, oslc_eca:futureAction points to oslc_eca:ActionDispatcher instead. oslc_auto:AutomationResult indicates that oslc_eca:Actions are available for immediate execution through the property oslc_eca:availableAction.

3.3.3. Rule Modelling

In addition to events and actions, it is necessary to have a model of automation rules to represent an ECA environment fully. These rules represent the conditions that must be met for an event to trigger an automated action. Figure 4 shows the relationship of the most important classes of the proposed model.

The oslc_eca:Rule class represents an automation rule and is described, as any other OSLC resource, by an oslc_eca:ResourceShape. It can point to an oslc_eca:Event with the property oslc_eca:triggeredBy and an oslc_eca:Action with the property oslc_eca:executes. The oslc_eca:consequenceOf property is also defined as actions that point to events that caused them to be executed.

Because OSLC is focused on interoperability, the proposal focuses on something other than how automation rules work internally. Depending on the service that manages the automation, the rules can be evaluated in many ways. They can even use one of the existing frameworks for semantic rules, such as SPIN [53] or Notation 3 Logic [54]. This would mean further interoperability, as they are also Linked Data standards. Therefore, the proposed rule modelling aims to establish a framework that can handle any inference engine based on the ECA model without specifying the method applied by the reasoning engine.

The oslc_eca:AssertedCondtion class provides a means to discover if some evaluation of a rule was successful. A boolean linked by the property oslc_eca:asserted indicates whether the evaluation was successful. An oslc_eca:AssertedCondtion points to the specific oslc_eca:Action generated with the property oslc_eca:produces when the rule condition is evaluated as true. The oslc_eca:asserts predicate links it to oslc_eca:Rule.

3.4. Example of Use of OSLC-ECA

In order to give a better idea of how specific events, actions, and rules are defined using the proposed OSLC-ECA model, this section shows a short excerpt of an example of the usage of these classes in a change management domain, i.e., using the OSLC Change Management Specification [46], following the situation shown in Figure 5.

The example begins with creating a bug/issue in a change management platform, such as Bugzilla [55], as shown in Listing 1. The following examples have been simplified for brevity, showing only the most interesting properties and relations.

Listing 1. An OSLC Resource oslc_cm: ChangeRequest representing the bug created in Bugzilla.

1   <http://example.com/ServiceProvider/sp1/bugs/b1> a oslc_cm:ChangeRequest;   2        oslc_cm:severity “enhancement”^^xsd:string;   3        oslc_cm:status “CONFIRMED”^^xsd:string;   4        oslc:serviceProvider <http://example.com/ServiceProvider/sp1>;   5        dcterms:contributor “admin”^^xsd:string;   6        dcterms:created “2022-12-10T07:07:33+00:00”^^xsd:dateTime;   7        dcterms:identifier “1”^^xsd:integer;   8        dcterms:modified “2022-12-10T07:07:33+00:00”^^xsd:dateTime;   9        dcterms:title “Testing bug to issue automation”^^xsd:string; 10        bgz:component “TestComponent”^^xsd:string; 11        bgz:operatingSystem “Linux”^^xsd:string; 12        bgz:platform “PC”^^xsd:string; 13        bgz:priority “---”^^xsd:string; 14        bgz:version “unspecified”^^xsd:string.

For this specific domain, some classes must be extended from the basis included in the proposal. Listing 2 shows a short example of an EventListener and a specific Event for this domain. In this example, we consider the EventListener to receive messages from a Kafka Broker when the bug is created in Bugzilla. Then, a BugCreationEvent individual is generated to represent this specific type of event.

Similarly to the specific types of events presented, Actions and ActionDispatchers classes must be defined for the exemplified scenario using the GitHub platform as the destination of the actions, as exemplified in Listing 3.

Listing 2. Example of use for oslc_eca:Event and oslc_eca:EventListener.

1        example:BugzillaEventListener a owl:Class;   2             rdfs:subClassOf oslc_eca:EventListener;   3             oslc_eca:binding [   4                  a kafka:Broker;   5                  kafka:topic “new_bugs_bugzilla”^^rdf:XMLLiteral   6             ].   7   8          example:BugCreationEvent a owl:Class;   9               rdfs:subClassOf oslc_eca:Event; 10               oslc_eca:generatedBy example:BugzillaEventListener. 11 12          <http://example.com/EventListener/el1> a example:BugzillaEventListener; 13               oslc:serviceProvider <http://example.com/ServiceProvider/sp2>; 14               oslc_eca:source <bugzilla.example.com>; 15               dcterms:description “Listener detecting events from  Bugzilla”^^rdf:XMLLiteral; 16               oslc_eca:binding [ 17                    a kafka:Broker; 18                    kafka:providerURL <http://broker.example.com/provider>; 19                    kafka:topic “new_bugs_from_bugzilla”^^rdf:XMLLiteral 20             ]. 21 22          <http://example.com/Event/ev1> a example:BugCreationEvent; 23               oslc_eca:generatedBy <http://example.com/EventListener/el1>; 24               oslc_eca:causedBy <http://example.com/ServiceProvider/sp1/bugs/b1>; 25               dcterms:created “2022-07-14T07:07:33+00:00”^^xsd:dateTime; 26               dcterms:title “Event generated by the creation of a bug”^^xsd:string.

Listing 3. Example of use for oslc_eca:Action and oslc_eca:ActionDispatcher.

1        example:CreateIssueGitHubActionDispatcher a owl:Class;   2               rdfs:subClassOf oslc_eca:ActionDispatcher;   3               oslc_eca:queryActions [   4                    a ldp:Container, example:GitHubActionQueryContainer.   5               ];   6               oslc_eca:execution [   7                    a ldp:Container, example:GitHubActionExecutionerContainer.   8               ].   9 10        example:CreateIssueGitHubAction a owl:Class; 11                 rdfs:subClassOf oslc_eca:Action; 12                 oslc_eca:executedBy example:CreateIssueGitHubActionDispatcher. 13 14          <http://example.com/ActionDispatcher/ad1> a example:CreateIssueGitHubActionDispatcher; 15               dcterms:description “Action dispatcher that creates issues on GitHub”^^rdf:XMLLiteral; 16               oslc_eca:execution [ 17                    a ldp:Container, example:GitHubActionExecutionerContainer; 18                    dcterms:description “REST API endpoint to create an issue”; 19                    github:executionURL <https://api.github.com/repos/example-repo/action-request-api>. 20               ]. 21 22          <http://example.com/Action/ac1> a example:CreateIssueGitHubAction; 23               dcterms:title “Action to be executed on GitHub”^^xsd:string; 24               oslc:serviceProvider <http://example.com/ServiceProvider/sp3>; 25               oslc_actions:executedBy <http://example.com/ActionDispatcher/ad1>; 26               oslc_auto:status oslc_auto:completed; 27               oslc_auto:veredict oslc_auto:passed; 28               example:originalChangeRequest <http://example.com/ServiceProvider/sp1/bugs/b1>; 29               dcterms:created “2023-04-14T07:07:33+00:00”^^xsd:dateTime.

Lastly, after the definition of Actions and Events, the definition of the Rule is addressed, as shown in Listing 4. In this example, the SPIN [53] framework has been selected as the inference engine, and some of its classes are included in the example.

Listing 4. Example of use for oslc_eca:Rule and oslc_eca:AssertedContidion.

1   2          example:BugsReplication a owl:Class;   3               rdfs:subClassOf oslc_eca:Rule;   4               rdfs:subClassOf spin:Rule;   5               oslc_eca:executes example:CreateIssueAction;   6               oslc_eca:triggeredBy example:BugCreationEvent;   7               dcterms:description “Create an issue in GitHub when an issue is created in BugZilla.”^^rdf:XMLLiteral.   8   9          example:NewBugCondition a owl:Class; 10               rdfs:subClassOf oslc_eca:AssertedCondition; 11               rdfs:subClassOf spin:Ask; 12               dcterms:description “Assert if there is any new bug created in BugZilla.”rdf:XMLLiteral. 13 14          <http://example.com/Rule/r1> a example:BugsReplication; 15               oslc_eca:triggeredBy <http://example.com/Event/ev1>; 16               oslc_eca:executes <http://example.com/Action/ac1>; 17               oslc:serviceProvider <http://example.com/ServiceProvider/sp4>  18               dcterms:created “2023-04-14T07:07:33+00:00”^^xsd:dateTime. 19 20          <http://example.com/AssertedCondition/ac1> a example:NewBugCondition; 21               oslc_eca:asserts <http://example.com/Rule/ev1>; 22               oslc_eca:produces <http://example.com/Action/ac1>; 23               oslc_eca:asserted “true”^^xsd:boolean; 24               dcterms:created “2023-04-14T07:07:33+00:00”^^xsd:dateTime.

Summarising, the examples in this section show a simplified usage of the OSLC-ECA model, available online [44] for a complete reference.

3.5. Prototype Architecture

This section presents a prototype architecture that uses the OSLC-ECA model as the foundation. This prototype architecture is a use case for the example presented in Section 3.4 and is not intended to be a reference implementation. It comprises the following four layers: the Service Layer, the OSLC Adaption Layer, the Distributed Messaging Layer, and the OSLC Automation Layer, as shown in Figure 6.

The tools and services involved in the automation scenario use case are located in the Service Layer. In the example presented in Section 3.4, these are GitHub [56] and Bugzilla [57]. In a real-world example, the set of tools could be as large and heterogeneous as required. Besides version control and bug tracking, many other types of tools are involved in the software lifecycle.

The resources of the services/tools are modelled and accessed through OSLC specifications to standardise communication among all services involved. Because many of the most popular commercial tools are yet to be integrated with OSLC natively, they are accessed through OSLC Adapters. These adapters translate service data from services to oslc_eca:Events and executes oslc_eca:Actions to real-world actions on a specific service or tool. Located in the OSLC Adaption Layer, these adapters consist of REST APIs compliant with the OSLC standard and a subscription/publication mechanism to send events or receive action requests.

The Distributed Messaging Layer consists of a message broker that implements some method to enable asynchronous communication between the services and the automation server, such as the publish–subscribe pattern, such as MQTT [43] or Apache Kafka [51], which allows distributed communication based on topics in real-world scenarios without restrictions of scalability or security.

Finally, the OSLC Automation Layer provides a standard communication interface with an automation engine, similar to the OSLC Adapters but not a source of events. The OSLC Automation Server implements the proposed OSLC-ECA model. It listens to oslc_eca:Events generated by the services and standardised by the OSLC Adapters. These events are registered in the rule engine, and finally, the resulting actions are sent back as oslc_eca:Actions to the services through the Distributed Messaging Layer. Behind the OSLC Automation Server, the rule engine will process the events, evaluate the stored rules, and return the corresponding actions. In the proposed prototype architecture, different automation tools could play the role of a rule engine, for example, StackStorm [14] or Clarive [15], using a simple extension that includes the events and actions involved in the scenario. However, a Semantic Rule Engine [58] can be used to maximise the advantages of Linked Data keeping the rule specification in a semantic format.

4. Results

This section shows the evaluation results of the proposed OSLC-ECA extension. The goal is to demonstrate that the model improves the current OSLC Automation standard by increasing its coverage of the features found in popular DevOps automation tools. The section is structured as follows. Section 4.1 shows the evaluation of the proposed model with coverage and similarity metrics. Section 4.2 presents a set of Competency Question (CQ) to exemplify the model’s adaptability. Finally, Section 4.3 discusses the results of the proposed model.

4.1. Completeness and Conciseness

Several methods have been used in the literature to evaluate ontologies. In [59], authors classify the different evaluation methods for ontologies into four categories: gold standard, corpus-based, task-based, and criteria-based approaches. A similar categorisation is presented in [60]: golden standard, data-driven, application, and criteria based.

The corpus-based or data-driven approach has been chosen for this paper. This technique is “used to evaluate how far an ontology sufficiently covers a given domain” [59], making it suitable for the task at hand. The features extracted from popular workflow automation tools conform to the corpus against which the semantic model is compared. The tools selected for the study are StackStrom [14], Clarive [15], GitHub Actions [24], and GitLab CI/CD [25], as explained in Section 2.1. An extensive analysis of these tools has been conducted to extract their list of features.

StackStorm and Clarive have been selected because they define automation that follows the ECA model. Clarive has many more features, but we focused on its event-based rulebooks. The semantic model has been designed following the same principles to make such tools OSLC-compatible. In addition, GitHub Actions and GitLab CI/CD have been chosen because they are trendy in DevOps automation scenarios. Although their workflows are not based on ECA, this evaluation shows that the semantic model can support them.

The features have been extracted by applying the algorithm shown in Algorithm 1 to the conceptual model of each tool. They have been classified into concepts, properties, relationships, and restrictions. This methodology has been used before by [61] to evaluate their semantic model. Then, because the semantic model extends OSLC, the list has been curated only to include features that affect interoperability. A sample extraction process for StackStorm rules is shown in

Table 3. Sample extraction from StackStorm model.

Feature	Type
∃ Rule	Concept
hasProperty(Rule, name)	Property
hasProperty(Rule, description)	Property
hasProperty(Rule, enabled)	Property
hasRelation(Rule, pack)	Relation
restriction(Rule, pack, range(Pack))	Restriction
hasRelation(Rule, trigger)	Relation
restriction(Rule, trigger, range(Trigger))	Restriction
restriction(Rule, trigger, maxCardinality(1))	Restriction
hasRelation(Rule, criteria)	Relation
restriction(Rule, criteria, range(TriggerParameter))	Restriction
hasRelation(Rule, action)	Relation
restriction(Rule, action, range(Action))	Restriction

.

Table 4. List of features extracted from each tool.

Feature	StackStorm	Clarive	GitHub Actions	GitLab CI/CD
Concepts	7	8	7	6
Relations	8	8	6	5
Properties	7	20	16	18
Restrictions	15	28	21	23

shows the number of features extracted for each tool and category.

Algorithm 1: Feature extraction process.

When evaluating ontologies, different possible criteria can be addressed. The authors in [59] define accuracy, completeness, conciseness, adaptability, clarity, computational efficiency, and consistency. For our proposed model, we evaluate completeness, conciseness, and adaptability.

Completeness is evaluated using the coverage function [62]. It measures the number of features in the corpus and quantifies how many are represented in the semantic model. A formal definition, with T being the tool and O the ontology:

$$Coverage(O,T)=\frac{Dim(O\cap T)}{Dim\left(T\right)}$$

In addition, conciseness is defined as “the criteria that states if the ontology includes irrelevant elements with regards to the domain to be covered” [59]. It is evaluated using the Jaccard similarity coefficient [63,64]:

$$Similarity(O,T)=\frac{Dim(O\cap T)}{Dim(O\cup T)}$$

With these two metrics, we can evaluate how the proposed semantic model approximates the features of each tool. Measurements have also been used to evaluate the OSLC Automation without the proposed extension to emphasise improvement.

Table 5. Coverage and similarity metrics presented by the proposed ECA model and OSLC Automation.

Tool	OSLC ECA		OSLC Automation
	Coverage	Similarity	Coverage	Similarity
StackStorm	0.86	0.74	0.4	0.27
Clarive	0.79	0.78	0.54	0.46
GitHub Actions	0.74	0.6	0.52	0.43
GitLab CI/CD	0.65	0.53	0.46	0.38

displays all the metrics described.

4.2. Competency Questions

The adaptability of the semantic model measures how it performs the task for which it was designed. To evaluate it, we present a simple case study with two commercial tools in which the model is used to implement automation. Then we estimate its adaptability using Competency Questions [65]. The scenario for the case study has been shown previously in Section 3.4, which includes two widely used tools in the industry: Bugzilla [57] and GitHub [56]. Bugzilla is used because there is already an OSLC adapter implemented for it [55]. Regarding GitHub, an adapter has been implemented specifically for this case study. It provides access to the GitHub issue platform and is based on the one used for Bugzilla. Finally, the OSLC Change Management domain [46] represents Bugzilla bugs and GitHub issues semantically.

With this scenario in place, the Competency Questions method can be used to evaluate the capabilities of a model to answer questions about its data. CQs can be formalized using a query language like SPARQL. We define five CQs that ask relevant questions about the system, as shown in

Table 6. Competency Questions to evaluate the adaptability of the semantic model.

Competency Questions	SPARQL Query	Result
Which rules are defined in the OSLC Automation Server?	SELECT ?description WHERE {       ?rule rdf:type oslc_eca:Rule;       ?rule dcterms:description ?description. }	“If a bug is created in Bugzilla, create an issue in GitHub.”; “If a bug is modified in Bugzilla, update the corresponding issue in GitHub.”
Which actions are available for execution on GitHub?	BASE <http://example.com/ServiceProvider/> SELECT ?description WHERE {       ?ad rdf:type oslc_eca:ActionDispatcher;       ?ad oslc:serviceProvider <sp3>;       ?ad dcterms:description ?description. }	“Create issue.”; “Edit issue.”; “Delete issue.”
What event triggered a rule, and what was its result?	BASE <http://example.com/Rule/> SELECT ?description ?result WHERE {       <r1> oslc_eca:triggeredBy ?event;       ?event dcterms:description ?description;       ?ac oslc_eca:asserts <r1>;       ?ac rdf:type oslc_eca:AssertedCondition;       ?ac oslc_eca:asserted ?result. }	“Bug created in Bugzilla.”, true
What was the source of a specific event?	BASE <http://example.com/Event/> SELECT ?source WHERE {       <ev1> oslc_eca:generatedBy ?el;       ?el rdf:type oslc_eca:EventListener;       ?el oslc_eca:source ?source. }	<bugzilla.example.com>

. First, we retrieve all the automation rules defined in the Automation Server. Then, we obtain the actions available for execution on GitHub. We prove that we can also acquire information about a specific resource, like what event triggered a particular rule and its evaluation result. Finally, we demonstrate how to find the source of a concrete event. None of these queries was possible with the previous OSLC Automation [23] specification.

4.3. Discussion

Established ontology evaluation methods have been used to compare the proposal with the existing OSLC Automation domain to evaluate the proposed model’s effectiveness and viability. First, coverage and similarity metrics are applied to measure the extent to which the proposed model covers the required concepts and how well it aligns with existing ontologies. Second, competency questions are used to gauge the model’s adaptability and assess whether the ontology can answer predefined questions. The results show that the proposed OSLC-ECA model can provide a standardised framework for service interoperability while addressing the vendor lock-in issue. The model adequately captures the necessary information, as demonstrated by a set of Competency Questions. These positive results support the validity and feasibility of the proposed event-based automation model for DevOps environments. Recapitulating, the proposed extension includes crucial aspects for Event-Condition-Action Automation in the current OSLC Automation specification. As demonstrated during the evaluation, the model shows sufficient expressiveness and adaptability to handle a real-life automation scenario such as those that software companies handle in their daily operation. Additionally, the flexibility the proposed approach allows is notable since the architecture used in the prototype (Section 3.5) does not establish what kind of rule inference engine or messaging system should be used. Thus leaving the freedom to use a non-semantic engine, such as StackStorm [14] or Clarive [15], or a semantic rule engine, such as EWE Tasker [66]. Similarly, we have the same freedom to use open-source messaging platforms for the communication layer, such as MQTT [43] or Apache Kafka [51], or any existing proprietary alternatives.

5. Conclusions

This paper presents an extension model for the OSLC standard to support ECA-based automation. It provides the concepts to make ECA automation possible in an interoperable OSLC environment. The end goal of this proposal is to use OSLC to facilitate integration and automation between services from different vendors or domains.

The proposal has been validated by studying the completeness and adaptability of the proposed model over a set of automation features extracted from popular automation-providing services. The model’s adaptability was demonstrated with a case study in which a real-life automation scenario was presented as a proof-of-concept example. A list of Competency Questions was provided to show the possibilities the extension allows. The contribution was also compared with the current OSLC Automation model using coverage and similarity metrics to measure completeness and conciseness.

The proposed OSLC ECA extension opens a new set of possibilities for automation management in DevOps environments. It allows model services as notifiers and workers controlled by predefined rules. This presents great flexibility for the OSLC specification since any new service can be easily integrated into the automation scenario, defining only the kind of events it can generate, actions it can execute and how to interact with it through a simple OSLC Adapter. Therefore, the contribution is expected to be helpful in complex automated environments where interoperability is a priority.

For future work, several paths present interesting potential. Since there is no specific inference process associated with the contribution, because the proposed model aims to define interoperable interfaces between services and the automation server, exploring semantic automation rules and bridging the proposed model with other ECA ontologies, such as Evented WEb (EWE) [61], and leveraging semantic web technologies can enhance the robustness and functionality of our proposal. This would integrate the proposed OSLC-ECA model into a semantic inference engine, such as EWE-Tasker [66], which would simplify the adoption of the proposed model. Furthermore, a methodology for adopting OSLC and ECA automation by DevOps practitioners could be highly valuable to facilitate adopting this approach in the industry and extending the available semantic specifications to new domains.

In conclusion, this article contributes to advancing open automation in software development systems by proposing an extension of the OSLC specification, enabling interoperability based on events, rules, and actions within a DevOps environment. It opens a new path for ECA automation in the OSLC specification. The validation results support the model’s effectiveness, and the exemplification of real scenarios illustrates its practical application.