The automotive industry is rushing towards new trends like autonomous driving and Car2X [1
]. Vehicles are becoming increasingly intelligent and interconnected with the outside world and with each other [4
]. The number of new functions in a vehicle is growing rapidly. Particularly in the context of autonomous driving, the need for special sensors that perceive the environment and complex sensor data fusion functions has increased. In addition to the actual control of the vehicle, other components such as driving assistants and infotainment gain in significance. As a consequence, the amount of mixed-critical data to be processed and exchanged has grown enormously.
The current state of the art is that each functionality is implemented on an adapted Electronic Control Unit (ECU). The ECUs are interconnected via proprietary communication buses like CAN and FlexRay [4
]. The rapid growth of data and interconnected functions encourage the automotive industry to look for more adaptive and high-end performance software platforms, as well as communication technologies that are able to achieve high throughput, provide determinism, and keep up with the dynamic communication pattern of new features.
The Automotive Open System Architecture (AUTOSAR) (https://www.autosar.org
, last accessed on 1 March 2021) is a global development partnership that addresses these software platform requirements with the ongoing development of the new standard AUTOSAR Adaptive Platform (AP). The new platform offers a high level of flexibility regarding software allocation, the underlying hardware, and network communication. AP relies on the dynamic deployment of customer applications and the use of highly parallelized processors. In comparison, the previous AUTOSAR standard, now called the AUTOSAR Classic Platform (CP), mainly encounters the requirements of deeply embedded systems [5
]. AUTOSAR Adaptive is designed to provide sufficient capacities for the processing of complex functions, e.g., image processing in Advanced Driver Assistance Systems (ADAS) and Automated Driving Functions (ADF), and to support distributed systems.
Moreover, AP is designed to fully support Ethernet-based communication. To better support automotive use cases, AUTOSAR specified the service-oriented communication middleware SOME/IP (Scalable service-Oriented Middleware over Internet Protocol) for the automotive/embedded field [6
] and especially for AP. Some contents of SOME/IP (events, fields, etc.) are even integrated in AP. Therefore, the Communication Management (CM) [2
] of AP already includes standardized bindings for SOME/IP, but also for the data-centric middleware protocol Data Distribution Service (DDS) [7
]. Besides the direct binding of a middleware protocol to CM, gateway solutions represent a further solution for the binding of additional middleware solutions [8
]. At the moment, (hard) real-time capable solutions exist neither with SOME/IP nor with DDS. Nevertheless, AP allows the specification of other direct middleware bindings, e.g., for OPC UA. The OPC UA technology supports real-time communication patterns. For a holistic real-time solution, a deterministic network technology like TSN is required on Open Systems Interconnection (OSI) layers 1 and 2.
The Open Platform Communications Unified Architecture (OPC UA) is a machine-to-machine (M2M) communication technology that is established in modern industrial automation. It is based on a service-oriented architecture (SOA). The focus of OPC UA is flexible communication using a Client–Server (C/S) and a Publish–Subscribe (PubSub) communication pattern to connect machines from different vendors. OPC UA satisfies the vertical communication covering all levels of the automation pyramid including field, control, supervisory, planning, and management level [10
]. Horizontal communication based on OPC UA is currently only established for controller-to-controller communication. In addition, OPC UA is seen as the key communication protocol in Industry 4.0 and IIoT (Industrial Internet of Things) [11
]. Additionally, OPC UA empowers connectivity to cloud solutions [12
] and is gaining popularity in smart grids [13
In 2018, the OPC Foundation started the Field Level Communications (FLC) initiative. Experts from global players like Cisco, Intel, Siemens, and TTTech join forces to establish OPC UA on the field level. This shall enable unified and vendor-independent communication among devices (sensors and actuators) and between field devices, controllers, and the cloud [14
]. Today, fieldbuses like PROFIBUS and Modbus as well as Ethernet-based protocols like PROFINET and EtherCat can be found on the field level [12
]. The FLC initiative aims to equip OPC UA with features required for industrial use cases (e.g., determinism, safety, security). Real-time capability is achieved with the OPC UA PubSub communication pattern in conjunction with Time-Sensitive Networking (TSN). In addition, the FLC initiative is developing methods for the configuration of Quality of Service (QoS) parameters, including TSN parameters [14
The Institute of Electrical and Electronics Engineers (IEEE) Time-Sensitive Networking Task Group (https://1.ieee802.org/tsn/
, last accessed on 28 February 2021) provides extensions to Ethernet networks defined in IEEE 802.1 and IEEE 802.3 to enable deterministic and reliable transmission of time-sensitive network traffic. TSN allows for mix time-critical network traffic with best-effort services. This so-called network convergence is particularly beneficial for the implementation of mixed-critical systems. The specification of a number of TSN standards has been completed while others are still in progress. The TSN Task Group has initiated several projects to define TSN profiles for different application domains such as industry, automotive, and aerospace. TSN is an upcoming technology which is only used sporadically today. One of the reasons is that there are not many TSN-capable devices on the market yet and that only few TSN mechanisms are supported. These mechanisms consist primarily of time synchronization and time-triggered transmission of data packets.
The combination of TSN and OPC UA enables flexible and deterministic communication for a wide range of devices from different manufacturers lowering the configuration overhead. Thus, it is not only interesting for industrial automation [10
], but also for other application domains like the automotive [18
] and railway industry [20
By integrating the AUTOSAR Adaptive Platform, OPC UA, and TSN, as shown in Figure 1
, the benefits of these technologies can be combined. This solution addresses the abovementioned challenges of the automotive industry. The goal of this paper is to show how these three technologies can be integrated and how determinism can be achieved. We give an overview of the mentioned technologies in Section 2
. In Section 3
, we review the related work. We present the integration architecture and discuss the impact of architectural decisions including crucial aspects of the implementation in Section 4
. Furthermore, we describe an experimental evaluation of the proposed communication architecture and discuss the results in Section 7
. Finally, we conclude our work and give an insight into future activities in Section 9
3. Related Work
The main contribution of this paper is the architecture for the binding between Adaptive Platform and OPC UA. Furthermore, we examine TSN as link layer technology. More specifically, we are presenting a network binding that exposes AUTOSAR service components (events, methods, fields) with the methodology of OPC UA. To the best of our knowledge, we are the first to propose a direct binding of the aforementioned technologies. While implementations of OPC UA over TSN already exist and numerous works have been published on this subject, the usage of OPC UA over TSN for real-time communication in AUTOSAR Adaptive is a novel concept. In the following, we want to provide an insight into literature that addresses the current state of the AUTOSAR Adaptive communication model, the benefits of the combination of OPC UA and TSN in factory automation, and OPC UA considerations in the automotive industry.
Regarding future automotive software platforms, AUTOSAR Adaptive is considered as a key technology to tackle the challenges of autonomous driving and Car2X in [1
]. The authors in [4
] state that AUTOSAR Adaptive will facilitate the development of a standardized platform for connected cars due to its wide range of platform services. While these works concentrate on the applicability and suitability of AUTOSAR Adaptive as a future vehicle platform, our research initially focuses on the binding concept of AUTOSAR Adaptive, OPC UA, and TSN and the evaluation of this combination for the in-vehicle communication. However, we also address the advantages of AP.
On the communication level, AP covers service-oriented communication with SOME/IP and DDS over Ethernet at the moment, but there is still a lack of a real-time communication scheme. TSN is such an option located on layer 2 of the OSI reference model. Although less dominant in literature, the automotive industry considers TSN a future trend in vehicle communication [3
]. The authors of [3
] identify TSN as appropriate technology for future car technology as it is expected to reduce wiring efforts. They clarify that, after chassis and engine, the wiring harness of a car is ranked third in both cost and weight. We have chosen to integrate TSN in order to be able to provide deterministic end-to-end latency for real-time communication in AP. The aforementioned advantages favor the use of the technology.
In the industrial domain, TSN is represented in many works [8
]. Since TSN is only represented at the lower layers of the OSI model, it is often considered together with a high-level protocol. The best-known representative in this case is OPC UA. Several publications regard the combination of TSN and OPC UA as future real-time communication candidate at the field level [10
]. TSN offers a variety of mechanisms such as EEE 802.1Qbv, IEEE 802.1Qch, IEEE 802.1Qcr, and IEEE 802.1Qbu to guarantee timely response [37
]. OPC UA, on the other hand, provides a connectionless and multicast-capable PubSub pattern and flexible configuration of data to be published [16
]. Ref. [32
] shows with example calculations that OPC UA over TSN outperforms current Ethernet-based fieldbus systems, especially because it allows the use of a Gigabit physical layer. In [16
], the authors evaluated the performance of OPC UA over TSN using the open-source implementation open62541 of OPC UA that supports C/S and PubSub. Our implementation also uses the open62541 stack due to the real-time performance considerations discussed in the following. The authors of [16
] showed that this PubSub implementation allows submillisecond publishing intervals with jitter in the nanosecond range. Others propose OPC UA PubSub for the delivery of TSN stream requirements to network components that are able to reserve and configure the necessary resources in TSN networks [37
]. OPC UA offers a rich information model that simplifies the configuration process [39
] and the discovery of publishers [37
]. Our work concentrates on the real-time capability achieved through the combination of both technologies. The configuration aspect is a future topic that we are planning to address, as mentioned in Section 9
To put the potential of OPC UA into context, researchers have compared the performance of open62541 with implementations of the middleware Robot Operating System (ROS), DDS, and MQTT in [40
]. Their benchmarks show a performance advantage of open62541 over MQTT and ROS implementations and a similar performance compared to the eProsima FastRTPS DDS stack. The authors additionally highlight the rich information model of OPC UA.
OPC UA is also a technology that is researched by the automotive industry. The thesis [18
] evaluates OPC UA for Car2X communication. The author concludes that OPC UA C/S brings considerable advantages for Car2X: It offers a suitable security model and facilitates interoperability. Furthermore, Ref. [9
] proposes a gateway enabling interoperability between VSOMEIP (GENIVI’s SOME/IP stack (https://github.com/GENIVI/vsomeip
, last accessed on 1 March 2021) and OPC UA applications to assure vehicle interoperability in the IIoT context. Such a gateway solution can also be applied to translate OPC UA C/S messages to SOME/IP Client–Server messages and vice versa in the AUTOSAR Adaptive context. The authors only worked with the Client–Server pattern and did not provide a direct binding of OPC UA concepts to SOME/IP or AUTOSAR Adaptive concepts, in contrast to our solution. In [8
], the same authors extend their gateway solution by an OPC UA PubSub and VSOMEIP Notify–Subscribe gateway application. On the link layer, the authors use raw Ethernet. The gateway application was tested in a small setup in which one or several OPC UA modules communicated with a VSOMEIP module over a gateway node. All applications were distributed on separate devices. The evaluation showed that the end-to-end transmission latency did not exceed the exchange interval chosen in the range of 1
. Our binding approach avoids the overhead of an intermediate application. It additionally embeds TSN as link layer and is able to reliably perform cyclic publishing at a 1
interval even in stress scenarios.
5. Binding Architecture
In the following subsections, we explain fundamental architectural concepts of our integration approach. We will go into detail about the mapping of AUTOSAR service components, types, configuration parameters, and service discovery to OPC UA methodology. Additionally, we discuss the approach for a binding implementation for real-time systems. The represented binding approach aims at combining the benefits from the adapted automotive platform AP, the rich information and communication model of OPC UA, and the real-time features of TSN.
5.1. Protocol Binding
Our concept integrates OPC UA C/S and PubSub inside a single binding. Both OPC UA facets provide access to a single information model for the AUTOSAR Adaptive services, see Section 5.3
. This covers the use case of simultaneous C/S and PubSub communication in the same application. For example, PubSub is suitable for real-time communication, while C/S provides access for diagnostics, management, and configuration.
shows that C/S and PubSub are integrated side by side in an AUTOSAR service provider. An OPC UA stack is placed inside the binding and contains the information model as well as the two protocol implementations. The information model is the shared data source for both protocols.
From the top-level perspective, the Adaptive Application provides data through mechanisms of the Adaptive Platform. The OPC UA binding inside the platform dispatches the data from the Adaptive Application into the information model, so that it is accessible via OPC UA C/S and PubSub.
From the bottom-level perspective, all service components (events, methods, fields) are accessible through C/S. PubSub is currently only used to transmit event data because PubSub does not match the concepts of methods and fields. The representation of the service components in OPC UA is described hereinafter.
5.2. Service Component Mapping
The three service components (events, methods, fields) are mapped to the concepts of OPC UA. The information model plays a central role as an interstation for data transferred between the network layer and the Adaptive Application.
With the existing bindings (SOME/IP, DDS), the call of the send method from the Adaptive Application triggers the transmission of a network message with event data. However, we chose a different approach for our binding. A send call from the application level only writes data into the OPC UA information model. The SourceTimestamp of the node is updated upon each write access. As AUTOSAR events are fire-and-forget notifications from the perspective of the Adaptive Application, the timestamp indicates the time at which the event was sent by the application.
The actual transmission of event data are realized separately. Figure 8
depicts the publishing mechanism for OPC UA PubSub: Once the provider applications send data (1), they are stored in the information model. A separate thread triggers a publishing routine at a configured interval (2). This means that data are retrieved from the information model and assembled into a PubSub network message. Cyclic publishing takes the place of event-based publishing for a couple of reasons. First, it addresses the requirements of cyclic (hard) real-time transmission as realized with TSN TAS. Second, the OPC UA PubSub specification [26
] assumes cyclic publishing as well. Third and last, cyclic publishing over PubSub allows for group events. All events of an event group are transferred inside a network message in each and every cycle. This reduces the overhead caused by protocol headers. A consumer application can detect whether an event was sent since the last cycle by comparing the included timestamps. Once the network message is ready for transmission, the publishing routine sends it to a multicast address (3). It is transmitted over a TSN-based network. Consumers that have subscribed to an event of the transmitted event group receive the message and record an event sample (4). In more detail, the binding unpacks event data and stores them in an event sample cache that is specific to the event. The Adaptive Application in the consumer can either poll the sample cache for new samples or register a callback to be notified about new samples.
As event data are stored inside the information model of an OPC UA server, clients can access them directly via C/S methodology. More specifically, clients browse through the information model of the server and read the variable node that belongs to the respective event. The client may be either a standalone OPC UA application or a part of a consumer’s network binding. Clients may also use so-called subscriptions [41
]. They are part of OPC UA C/S methodology (not to be confused with PubSub). Subscriptions allow clients to subscribe to data points. The server then monitors these items and notifies clients about changes.
AUTOSAR Adaptive methods are remote procedure calls (RPCs). A service consumer calls a method with a list of parameters and, optionally, receives a return value with the result. This concept can be mapped one-to-one to OPC UA method nodes. An OPC UA client calls a server method by passing input arguments. The OPC UA server uses a callback mechanism to run the function registered by the Adaptive Application. After completion, the server returns a list of output arguments.
Methods are only implemented for C/S access. We do not intend to realize them using OPC UA PubSub since PubSub does not support a matchable concept.
An AUTOSAR field has, unlike an AUTOSAR event, a certain value at any time. Conceptually, it inherits features of methods and events. The value of a field can be accessed using a getter method. Writable fields contain an additional setter method. Consumers can subscribe to fields. When a field value is updated, it notifies subscribed consumers with the same mechanism as regular AUTOSAR events. We decided not to map fields in the same way. Instead, we realized fields as variable nodes in the information model. The access level of the variable node specifies whether the node can be written by a client. This substitutes the existence of getter and setter methods. Clients can subscribe to it using the OPC UA subscription mechanism. Notifications triggered by the subscription substitute the functionality that is realized as event in SOME/IP and DDS.
More technically, variable nodes for fields have an attached data source. This means that the nodes do not store the value themselves but manage callbacks that handle read and write access. Figure 9
visualizes this concept. The OPC UA server of a provider redirects every read and write request to a callback function. The particular callback function is the get handler or set handler of the binding, respectively. This handler, again, calls a get or set callback registered by the Adaptive Application. From the consumer perspective, the OPC UA client within the binding writes or reads the respective variable node inside the server information model. Event data are directly transmitted between the Adaptive Application and the OPC UA client through the ara::com interface. The OPC UA client does not have an information model.
Subscriptions regularly use sampling (polling) to detect value changes. This would lead to a performance penalty since it would trigger the get callback of the application periodically, e.g., at an interval of 50 ms. We recommend to use OPC UA’s so-called exception-based model [41
] (Section 188.8.131.52) instead. This concept avoids sampling and requires the overlying logic (here: the binding) to notify the OPC UA stack about value changes proactively. The subscription then triggers a notification to subscribed counterparts.
Fields are only implemented for C/S access. We do not intend to realize them using OPC UA PubSub since PubSub does not support a matchable concept.
5.3. Deployment Parameters Mapping
AUTOSAR specifies a development methodology for Adaptive Applications. This includes a standardized approach to describe components using the AUTOSAR manifest, see Section 2.1.2
. The manifest section that specifies how a service and its components are mapped to a specific communication technology is called ServiceInterfaceDeployment, see Figure 10
. For example, it assigns protocol-specific numeric identifiers to services or events and provides structures and parameters to group AUTOSAR events for joint transmission. A service interface can be instantiated multiple times with ServiceInstanceDeployment. The resulting service instances are derived from a protocol-specific ServiceInterfaceDeployment. The ServiceInstanceDeployment contains parameters and concrete configurations specific to a service instance. Service instances can be again mapped to machines to obtain communication configurations in a ServiceInstanceToMachineMapping. The ServiceInstanceToMachineMapping maps e.g., multicast IP addresses to service instances and assigns communication connectors of the machine to the instance. We designed a ServiceInterfaceDeployment and a ServiceInstanceDeployment for OPC UA. Furthermore, we mapped the parameters of the deployments to concepts available in OPC UA.
We have integrated the AUTOSAR service interface artifacts as a custom-designed OPC UA information model. Figure 11
shows the representation of a service instance in the information model from the perspective of a client. It is the AUTOSAR service representation of the exemplary camera sensor introduced in Figure 5
. In more detail, a folder Services is placed inside the preexisting Objects folder of the OPC UA address space. The Services folder contains object nodes for every service interface offered by the provider (here: Camera_Service_Interface). Each service interface object contains OPC UA variable nodes with a numeric identifier (ID) and numeric version information (Major, Minor) of the service interface. Additionally, this object contains another object for every service instance of the respective interface (here: Camera_Instance_1). Service instance objects, again, have a numeric identifier and folders for the service components (events, methods, fields). Objects for specific service components exist inside each folder. These objects have two nodes: a numeric identifier variable (ID) and the actual content of the component (Value, Method). In this example, we represent the status of the camera as an event, the configuration routine as a method, and the image as a field.
The object nodes mentioned above are derived from custom object type definitions (ServiceInterfaceType, EventType, etc.). To promote interoperability, these definitions can later be consolidated in a specified OPC UA information model, called Companion Specification.
The representation of AUTOSAR events in OPC UA PubSub is based on PubSub-specific configuration structures. Figure 12
shows the configuration structures involved in the PubSub publishing process, including a selection of parameters. Our mapping specifies how parameters from the AUTOSAR interface deployment are mapped to these OPC UA configuration parameters. The PubSub configuration model exposes a lot of optional configuration parameters that we did not use for our mapping approach. Particularly relevant structures are PublishedDataSet, DataSetWriter, WriterGroup, and PubSubConnection. DataSetWriters generate so-called DataSetMessages based on parameters provided in a PublishedDataSet. In our mapping, a DataSetMessage can contain the data of a single or several event samples. The DataSetMessage contains further parameters like type encoding. In more detail, the DataSetWriter processes data extracted from the information model, called DataSet in the PubSub context. DataSetWriters complement DataSets with an identifier DataSetWriterId which contains the numeric identifier of an AUTOSAR event in our mapping. As explained before, events can be grouped. This is done using WriterGroups that aggregate multiple DataSetMessages. The result is a NetworkMessage. A WriterGroup adds the WriterGroupId that we use to store an identifier of an event grouping. Furthermore, the WriterGroup is responsible for cyclic publishing. It is therefore configured with a publishing interval. The PubSubConnection is the configuration structure for the network transport. It contains the PublisherId which is a unique identifier of the publisher. We use this field to store the numeric identifiers of the AUTOSAR service interface and service instance. For example, the PublisherId has a length of 32 bits. The 16 bits interface identifier is stored in the upper half of the PublisherId, the 16 bits instance identifier in the lower half. Additionally, the PubSubConnection stores the destination address of the PubSub messages (in our case a multicast address).
shows a summary of the mapping described above. Configuration structures of OPC UA are mapped to AUTOSAR concepts. An OPC UA binding contains a single UA_Server. A UA_Server contains a variable number of PubSubConnections (in open62541). A PubSubConnection is created for each ServiceInstanceToMachineMapping. For every EventGroup in AUTOSAR a WriterGroup is added to the PubSubConnection. In addition, finally, a DataSetWriter is added to the WriterGroup for every corresponding EventDeployment in AUTOSAR. A PublishedDataSet is created for the parameterization of each DataSetWriter.
5.4. Type Mapping
Key functionality of the binding is type mapping. A consistent strategy is required to convert AUTOSAR-level data into OPC UA-compatible data that are eventually serialized into an on-wire byte stream. The consumer side must have the same understanding of data types to properly deserialize the byte stream and convert it into high-level AUTOSAR-compatible data. For this purpose, OPC UA provides a comprehensive data modeling framework that can be exploited in different ways.
AUTOSAR supports basic types (Integer, Float, etc.) as well as structured types which are compositions of basic types. A general constraint is that instances of these structured types must be written in one transaction, i.e., as a whole. It must not be possible to modify single structure elements separately. Therefore, it is not possible to map structure elements to separate OPC UA variable nodes. The following sections propose two options for type mapping. The first option is based on OPC UA container type variant. The second option makes use of custom data types.
5.4.1. Variant-Based Mapping
Variant-based mapping makes only minimal use of OPC UA’s rich data modeling. This strategy is based on variants, a generic container type. Variants can hold any other data type as scalar or as an array [43
]. This facilitates to map structured data types in AUTOSAR to nested variant arrays in OPC UA. It follows the paradigm “Everything is in a variant”.
The variant-based mapping completely decouples providers and consumers on OPC UA level. No explicit exchange of OPC UA data types is necessary. As a result, this approach does not ensure type safety on an OPC UA level. The OPC UA stack only processes the variants and passes data values to the higher level AUTOSAR logic. Type safety and information interpretation are realized on an AUTOSAR level. Therefore, applications that do not run on top of AUTOSAR must perform type binding on application level.
5.4.2. Custom Data Type Mapping
This option especially covers the use case of communication between AUTOSAR and non-AUTOSAR applications. OPC UA applicants regularly make use of custom type modeling to represent complex data structures in the address space. That raises the need for the binding to support type mapping based on custom type definitions. The usage of custom types must be fully transparent to any AUTOSAR application.
In more detail, the OPC UA information model contains a set of predefined data types. Among them is the subtype Structure, which, again, contains subtypes. Model designers can create custom subtypes of Structure. The subtypes have an attribute of type StructureDefinition. This attribute contains (among other values) an array of elements that describe the single structure elements. OPC UA transfers values of Structure data types in a byte string. To decode the byte string, the information defined in the StructureDefinition is needed.
Several options exist for the exchange of data type definitions. They differ in the time of exchange. Consumer applications can receive type definitions at compile-, startup-, or at runtime. Compile-time exchange is realized by compiling and linking source files with type definitions into consumer applications. When using startup-time exchange, the consumer reads a binary file containing type definitions. Runtime type exchange can be realized with C/S communication between consumer and provider. Setting up a central directory for type definitions is another option. In [44
], we are presenting details about type exchange options.
5.5. Service Discovery Mapping
As its name already suggests, AUTOSAR Adaptive focuses on the flexible and dynamic interaction between vehicle components. Service consumers are capable to find matching service providers. The Adaptive Platform allows communication path establishment at design-, startup-, or runtime. For the two latter approaches, a central service registry is specified in the AUTOSAR standard. The registry implements three discovery operations: Service providers announce their service with Offer and withdraw it using StopOffer. Subscriber applications query the registry for a specific Service by calling Find. The registry concept itself is abstract and protocol-agnostic. It must be mapped to a specific protocol.
We evaluated OPC UA discovery mechanisms for usage as service registry. OPC UA specifies two discovery mechanisms: Local Discovery Server (LDS) and Global Discovery Server (GDS). The LDS keeps track of OPC UA servers running on a host system and exposes them publicly. The GDS, on the other hand, can be used for discovery across multiple subnets. Since intersubnet discovery may be required by certain use cases, we chose the GDS for the role as registry. A GDS exposes an information model that contains UA methods for registration and query of OPC UA applications [45
Since not all properties of an OPC UA application match the properties of an AUTOSAR service, we designed an explicit mapping of service properties to properties of OPC UA applications. The GDS provides certificate management services, too. This capability may be exploited for encryption and authentication.
In a later phase of our research, we found that some requirements cannot be fully met with a GDS-based service registry. Two major shortcomings had an impact. First, the GDS does not implement a concept that corresponds with the TTL (time to live) of an AUTOSAR service. Second, the GDS does not allow domain specific extensions. This means that additional domain specific attributes cannot be attached to an application record. For these reasons, we decided to come forward with a proposal for an alternative registry concept. This custom service registry is a domain specific OPC UA server. In the binding specification, we describe an information model with data types (e.g., ServiceRecordDataType) and methods (e.g., OfferService). Both directly map the AUTOSAR concepts.
Beside these centralized discovery approaches, a distributed option is introduced in the AUTOSAR standard [46
]. With the distributed approach, a single responsible registry is not needed. The registry information is distributed among participants via broadcast communication. Applications hold a self-administered view of the registry. We are seeking to implement the distributed discovery using OPC UA PubSub in the future. Therefore, we added data types to the custom service registry that represent registry operations (i.e., Offer, StopOffer, Find). Instances of the types are transmitted inside PubSub messages. While being specific about data types, we left the publisher configuration (e.g., WriterGroups, DataSetWriters) unspecified. If a domain with a single responsible registry and a domain with distributed registry need to be connected, a discovery bridge can link these domains [46
]. The discovery bridge also holds a local registry and acts as a broker between the domains.
6. Binding Implementation for Real-Time Systems
We developed the OPC UA integration (binding) based on the AUTOSAR reference implementation version 19-03. It is written in C++. With respect to OPC UA, we decided to use the open-source implementation open62541. It is written in the C programming language. Its source code is available on GitHub. We used the GDS implementation that is available with .NET Standard Stack from the OPC Foundation (https://github.com/OPCFoundation/UA-.NETStandard-Samples/tree/master/Samples/GDS
, last accessed on 1 March 2021). Microsoft’s .NET Core allows running .NET applications on diverse platforms, including Linux. The following subsections explain noteworthy aspects of the implementation.
6.1. Multithreading Support
For the sake of responsiveness and execution speed, the AUTOSAR platform uses multithreading. In general, real-time systems must implement measures to guarantee integrity and determinism in multithreading scenarios. For this purpose, open62541 introduces different levels of multithreading support. With activated multithreading support, API calls (e.g., UA_Server_read and UA_Server_write) are mutually exclusive. This means that a write operation cannot be preempted by a read operation. For a deterministic PubSub-only publisher, we assume two threads: a publishing thread with read access to the information model and an application thread with write access to it. The time-critical functionality of both threads would be mutually exclusive. One thread would have to wait for the completion of the other. For determinism, both threads must have reentrant (i.e., lock-free) information model access. Hence, the aforementioned multithreading mechanisms are inapplicable.
Contributors of the open62541 project propose a lock-free mechanism that guarantees a consistent view on the information model and data integrity. What they describe in [16
] is a copy-and-replace technique: The underlying data structure of the information model is a hash map pointing from the node identifier to the node representation in memory. Each node is treated as immutable, i.e., it cannot be modified once it has been inserted into the hash map. For modifications, it is possible to replace the node with a modified copy. The replacement is performed using an atomic compare-and-swap (CAS) operation. This way, the information is always consistent, even if a write operation is interrupted mid-update. Information model access can be reentrant. The implementation of this mechanism comes with the open62541 major branch.
The binding implementation uses this concept. However, since the binding is a complex scenario that requires deterministic behavior in a multithreading environment, it requires further measures. In the following subsection about real-time memory management, we build upon this concept and extend it.
6.2. Real-Time Memory Management
The implementations of the Adaptive Platform and open62541 make recurring use of dynamic memory management, i.e., heap allocations with malloc/free in C or new/delete in C++. The paper [47
] points out the following problems with regular dynamic memory management: sufficiency (refusal of allocation), memory leaks, fragmentation, and timeliness (undefined timing behavior).
For these reasons, it is highly discouraged to use dynamic memory management in real-time systems requiring deterministic behavior. Some embedded systems avoid the use of dynamic memory management, and thus the heap, altogether [48
]. However, both the Adaptive Platform and OPC UA are in nature dynamic. AUTOSAR Adaptive requires dynamics for its scalability. OPC UA information model has a database character. Hence, it is not possible to eliminate dynamic memory management in the whole application.
We tackled this problem by eliminating dynamic allocations only in time-sensitive paths. As Figure 14
shows, dynamic memory management is restricted to the system startup phase. Buffers that are needed for real-time operation are preallocated. A freeze event introduces the real-time phase in which the buffers are used.
In more detail, the complete program path reaching from the platform method for sending an event to the actual PubSub message generation logic must be examined. Timely generation of network messages is especially crucial in conjunction with TAS setups with cyclic transmission of real-time data. If a packet is not ready to be sent when a cycle starts, it must wait until the next cycle.
In a first step, we replaced all dynamic containers (like the STL container vector) with preallocated custom containers. Using a standard STL container with a custom allocator, as shown in [49
], would have been possible as well.
In a second step, we extended the concept of immutable nodes described above. It is problematic that a new node is allocated upon each write access to a node. We decided to preallocate two buffers for each node that stores event data. Upon node modification, the node currently referenced by the hash map is copied to the other buffer. Once the modification is complete, the reference in the hash map is changed to the other buffer (buffer alternation).
In the last step, we handled the remaining dynamic allocations in the message generation functionality. It is beneficial that all memory allocation during the generation of a PubSub message is freed after the message is sent. Thus, we could redirect all allocations within the message generation to a preallocated buffer. Knowing that none of the allocated memory is needed after sending the message, the subsequent publishing routine can write into the buffer again starting at address 0.
An alternative to the restriction of dynamic memory management is the usage of a real-time memory allocator. Such allocators pursue to overcome the variable runtime and fragmentation inherent to regular allocators. One example is the TLSF allocator described in [50
]. Allocations have constant runtime, i.e.,
. A drawback is that current implementations do not support thread safety. It must be provided by the integrator.
6.3. System Level Synchronization
Event data generated on the application level of a service provider have to pass through different system levels until it is finally transmitted on wire. Figure 15
shows these different levels and data flows (blue lines). The yellow dashed line visualizes the data flow of an event: Data are sent from the application level to the OPC UA information model using methods of the Adaptive Platform (specifically ara::com). The information model is a buffering stage that holds data in nodes. The separate publisher thread accesses the nodes and assembles a network message that is sent through an operating system socket. The Ethernet TSN network interface, again, is a buffering instance that holds packets until the start of the assigned time slot (see Section 2.3
At the moment, scheduled cyclic execution of Adaptive Applications is not yet realized as platform functionality. For this reason, we used oversampling on the publishing level in order to reduce event sample loss. This means that the publishing interval is shorter than the assumed minimal send interval of the application. Figure 15
shows a factor 2 oversampling, i.e., the publishing is triggered twice as often as the application sends data.
The TSN adapter and the application space must have a common notion of time. This is achieved by synchronizing a system clock (here: CLOCK_REALTIME) to the adapter’s PTP Hardware Clock (PHC), which itself was synchronized with the PHC of other TSN participants. It is thereby possible to start the generation of PubSub messages at a specified time relative to the TSN cycle start (offset). This method cannot only guarantee that a network message is ready for transmission upon cycle start, but it also ensures that the data are up-to-date. The cyclic publishing is technically realized within a dedicated thread. Its core is a loop that in each iteration waits for the next scheduled cycle (using POSIX method clock_nanosleep), executes the publishing, and calculates the next wakeup time.
It is also possible to introduce a second offset that synchronizes the application-level data provisioning and the OPC UA publishing. The Adaptive Applications sends event data at a specified time before the publishing routine reads data from the information model. One way to realize this is by using the POSIX method clock_nanosleep in the implementation of an AUTOSAR application. It should be noted that this behavior is not part of the AUTOSAR platform. It must be implemented by the application programmer.
6.4. Real-Time Scheduling Considerations
To achieve system-wide real time, deterministic network transmission is not the only requirement. Holistic system design must also cover determinism on the application level. In the AUTOSAR environment, Execution Management (see Section 2.1.2
) allows the configuration of scheduling and resources. It works together with the operating system to initialize the runtime scheduling of applications [5
]. The Deterministic Client, as introduced in the Execution Management Specification [21
], is supposed to support the event and time-triggered execution of Adaptive Applications in the future. It offers a deterministic worker pool, distribution of activation time stamps to Adaptive Applications, and random numbers generation [21
]. Not all details about the Deterministic Client are given yet. EM is not responsible for the performance of the scheduling. It is the responsibility of the operating system. In order to provide real-time resources to applications, the person or the tool integrating applications into platforms and machines is in charge of allocating enough resources, assigning appropriate scheduling policies and priorities, and monitoring deadlines. The configuration of resources using the support of the Execution Management can be specified in the AUTOSAR manifest. In this case, scheduling parameters in the manifest are based on the operating system interface POSIX (IEEE 1003.1 [51
]). Real-time scheduling policies SCHED_FIFO (First In, First Out) and SCHED_RR (Round Robin), as well as non-real-time policy SCHED_OTHER can be selected as valid scheduling policies for applications using EM [21
]. Other scheduling policies such as SCHED_DEADLINE are not prohibited, but they may not be portable across different AP implementations [5
]. The scheduling policy SCHED_FIFO allows realizing scheduling algorithms such as the rate-monotonic scheduling with fixed priority assignments [52
] (p. 354).
One prerequisite is the reliable determination of the worst-case execution time (WCET) of an application that is an important metric for schedule calculation. The operating system strongly affects the WCET as it introduces task scheduling overhead, interferences between running tasks, and system call execution. Unfortunately, in general-purpose systems with real-time extension (e.g., Real-Time Linux), the upper bound of the WCET is difficult to estimate and may be too pessimistic compared to special-purpose embedded systems [53
Another prerequisite is the ability to preempt the execution of threads, e.g., as provided in the Linux PREEMPT_RT patch (https://wiki.linuxfoundation.org/realtime/start
, last accessed on 18 March 2021). High priority threads must be able to preempt running low priority threads. The fixed assignment of priorities complicates the addition of new threads with latency bounds at runtime. As a result, the old schedule may become invalid and applications could miss their deadlines. Thus, all applications demanding tight latency bounds should be known and considered before runtime to achieve an optimal or near-optimal schedule.
As AP promotes flexibility and scalability, distribution of applications across different AUTOSAR platforms and machines is possible. For distributed cyclic real-time applications, it means an increased complexity to create a feasible schedule to satisfy all end-to-end latency requirements. The end-to-end latency of communicating applications comprises the execution of the provider applications, the transmission of the messages over IPC or a TSN network in our case, and the execution of the consumer applications. If the network is involved, both the network and the task schedules have to be coordinated. One solution to address task and TSN network scheduling has already been investigated in [54
]. In future works, we will elaborate on the subject matter in more detail.
6.5. TSN Configuration and Parameter Mapping
During our research, the OPC Foundation was working on a specification for a configurable transport mapping of OPC UA PubSub to TSN. The specification was in draft state and not publicly available. This means that a generic mapping solution as it would be useful for the AUTOSAR network binding was not yet available. The configuration of TSN (TAS) must be done under consideration of the holistic system design, anyway. This includes the definition of streams for different service classes, e.g., real-time or best-effort. TSN scheduling configuration is a field of its own that is currently under investigation, as discussed in our previous work [29
]. Specific capabilities of the employed TSN devices must be taken into account as well. Temporarily, we set the TSN configuration manually. This includes the setup of time synchronization (e.g., PTP), VLAN configuration (VLAN ID and priority mapping) as needed by IEEE 802.1Qbv and the installation of a TSN schedule (cycles, slots).
By proposing an integration approach of the AUTOSAR Adaptive Platform, OPC UA, and TSN, we presented a holistic solution for modern vehicle communication. Particularly, we showed that a combination of OPC UA C/S and PubSub qualifies as middleware for the Adaptive Platform. Our main contribution is the binding concept between the Adaptive Platform and OPC UA that harmonized the architectures of both technologies. We proved the concepts with a prototype and presented notable tweaks of the implementation. Furthermore, we used this implementation to deploy a TSN test setup and verified a deterministic communication schema in an end-to-end scenario. However, as our evaluation shows, the scheduling effort of AP applications, OPC UA PubSub, and TSN should not be underestimated.
The merit of the AUTOSAR Adaptive platform is its development methodology that enables efficient software engineering. By using OPC UA as a communication protocol, the Adaptive platform adopts a widespread standard that provides a rich information and communication model. This allows manufacturers to purchase commercial off-the-shelf components and avoids vendor lock-in. Similarly, TSN is expected to be implemented by network components that are available in a competitive market. The convergence of real-time and best-effort traffic in a single physical network saves costs and weight. However, further work needs to be done to promote determinism in Adaptive Applications. Time and data determinism must be guaranteed using elaborate resource management and scheduling. The AUTOSAR Deterministic Client [21
] aims to address these requirements.
In addition, the TSN standardization is still in progress. Thus, there is still a lack of appropriate and ready-to-use TSN hardware to realize reliable automotive systems. Furthermore, the configuration of TSN-capable devices has not yet been fully investigated. The configuration aspect of TSN devices will not be part of the TSN standardization [35
]. There is especially a need for concepts to configure end-systems. However, some papers have already addressed this problem by proposing OPC UA as a possible solution [32
In the long run, we pursue the integration of our binding specification into the official AUTOSAR Adaptive release alongside existing network bindings. This step would address the rising relevance of OPC UA and TSN across domains.