A graph-based, semantic system modeling approach forms the foundation of the toolbox. Based on this foundation, several software tools and graphical user interfaces (GUIs) were developed to create, modify, and deploy robot designs.
2.1. System Modeling and Database
The modeling layer is the core of the software framework, consisting of the xtypes-generator, xtypes, and xdbi libraries. The xtypes-generator defines the XType base class, which stores the properties and relationships between XTypes and generates project-specific models.
The
xtypes library builds on this by defining key classes for modeling robotic systems (an overview of the classes and their relations is given in
Figure 1, and the modeling mechanism is described in more detail in [
8]).
The core class is ComponentModel, which represents the types of components and (possibly) their inner structure, while Components are specific, configured instances of these types. Similarly, InterfaceModels describe the types of interfaces, whereas Interfaces represent their instances. By connecting Interfaces e.g., the dataflow in a software network, the assembly information of a physical entity is encoded. Networks of Components and Interfaces can be reused across different systems, forming new ComponentModels.
XTypes also support the combination of abstract and concrete elements in system design. Templates can be created with abstract components, serving as placeholders for specific parts to be resolved later. This allows users to start with a flexible, high-level structure and to add concrete details when necessary.
Modules are created from ComponentModels when, e.g., a real physically existing robotic system is built from it, which is an example of system architecture description. When they are created, all of the abstract components inside are resolved and their configuration can only contain fully specified values. Therefore, they represent the so-called digital twins of their real-world counterparts. In software, this means that such a module is a digital twin of the running tasks inside the robot; in hardware, a module would be a digital twin of one of the actuators, e.g., the rear-left hip joint.
DynamicInterfaces represent interfaces, whose existence depends on the Component’s configuration and are transformed to Interfaces once the configuration is valid.
ExternalReferences link ComponentModels to additional resources like manuals, CAD data, simulations, or their actual software implementations.
The xdbi layer defines the interface and connects to the XRock-database, which stores XTypes and their relationships.
Multiple source databases and their order in lookup can be specified, but there can only be one destination database (where modifications can be made). This enables multiple users to access, store, and retrieve system designs, both locally and remotely, via REST API, thus facilitating the easy sharing and reuse of components.
2.2. Workflow and Tooling
The workflow for designing a modular robotic system based on the toolbox can be depicted as the development cycle that is visible in
Figure 2, with the final result being in the center. In the current state, the Robot Construction Kit (
ROCK) is used as the source of software components and the backend to deploy the final runtime software [
9]. However, most of the tools are framework-agnostic, so other frameworks can also be supported. In the following section, the developed tools are presented in order of the usage in the workflow cycle.
We start with the Component Design step, which lays out the new software and hardware components that are required for the target system (if they do not exist yet). The hardware models are created using Phobos, an open-source Blender add on, while the software component models are built with help of the oroGen tool set. Phobos allows users to annotate kinematic and visual models from CAD software with essential details like sensors, collision shapes, and interfaces. The new component models, including all relevant information, are added to the database using xdbi.
For special purposes, other tools for component design exist to support and simplify the creation of generic atomic ComponentModels and their interfaces, as well as to link abstract components to their concrete implementations. Abstract components are used to create model templates, which are resolved (e.g., by user choices) whenever a concrete version is requested.
Using XRock-GUI or XDashboard, the needed components are evaluated and selected for further composition in the Component Selection step. Since each component model, including both software and hardware, is stored in the database with its properties, the system designer is able to find the ones that best fit for the requirements. XDashboard is a central web interface where users can access all of the tools needed for creating and setting up components. Based on where they are in the process, tools like the XRock-GUI, Phobos, and other tools can be accessed right from the dashboard. New tools can be added easily as plugins, making the system flexible and expandable. The integrated Component Browser connects to the database where users can view and edit the entries. It lets users combine parts to create simple new models, allowing for the easy expansion of the system without complex connections. XRock-GUI is a desktop application that facilitates the creation and configuration of components within a new ComponentModel.
The components and their interfaces are displayed as editable nodes, allowing users to easily modify connections. The GUI ensures valid connections and supports configuration changes, including the global variables resolved at deployment, reducing manual edits and ensuring consistency.
Within the Composition step, the selected component models are composed into the new robotic system’s model to form both the hardware structure (mounting points and orientations, joints, and rigid structures), as well as the software needed to control it. Closely related is the Configuration step, in which the different components are configured to work together. This means that the software interfaces are connected, hardware properties updated, etc. The hardware composition utilizes the Deimos tool within XDashboard, while the software composition and configuration is conducted through XRock-GUI or XDashboard. Deimos is a 3D web app that helps users assemble and align hardware parts in a ComponentModel. This makes it easier to ensure all of the parts are correctly positioned relative to one another.
After the Analysis of the built architecture (such as, for example, the validity of connections, workspace analysis, etc.) an assembly manual for the hardware, as well as a compiled and executable software application, is constructed. In cases where the robotic system is to be simulated, the required artifacts—like URDFs and visuals—are generated, and the necessary application is compiled to execute it.
Finally, during the Integration phase, either the physical robotic system is constructed or the complete setup is run in the simulation, such as, for example, with MARS. Based on the tests with the robotic system or in simulation, there might be a need for adaptations, in which case, the workflow cycle starts anew.
To design and plan missions based on the available robotic systems and other objects,
Mission-Planning was integrated, which triggers multiple phases of the workflow cycle. Based on the
Moreorg ontology, the required capabilities and provided services of
ComponentModels are specified within the
xtypes ecosystem. The
TemPl planning algorithm is integrated to choose the suitable systems for a mission and to provide a spatio-temporal plan, e.g., such as determining when a specific system has to perform certain tasks at a specific location or with certain objects.
TemPl also plans the online reconfiguration of robotic systems to minimize the resources used, the load to carry, and the distances to travel. Both
Moreorg and
TemPl are described in detail in [
10].
2.3. Hardware Toolbox Components
To reduce the costs and time needed when developing new systems, a toolbox with hardware modules was developed. A system can be configured and set up using these modules, and third-party components can be integrated. The modules can be connected to each other and to the third-party components via the so-called inter-modular interface (IMI) (see
Figure 3g) and via the electro-mechanical interface–modular (EMI-MOD) (see
Figure 3b,c). The IMI allows for a fast manual connection and disconnection between two components, and the EMI-MOD enables the online reconfiguration of the system to fulfill a variety of tasks and even extend its functionality by adding new modules or components as soon as they are equipped with at least one interface. Sustainability is ensured by the fact that defective modules can be quickly replaced via the EMI-MOD using the manipulator arm.
As one of the core elements, a modular 6 DOF
manipulator arm was built *see
Figure 3a). The design here directly follows the toolbox systematics, such that the manipulator is constructed from various functional units. The end effector of the arm is equipped with an EMI-MOD on both sides. This allows for the location of the manipulator to be changed during operation by bridging over from EMI-MOD to EMI-MOD and even changing the deployment system and its end effector payload according to the mission requirements. In MODKOM, the largest available version was built with a total length of 1628 mm, a mass of 15 kg, two pan tilt units with a 300 Nm peak, and an effective payload of 7 kg (a max. of 20 kg is theoretically possible). All of the joints were capable of 360° movement.
The
EMI-MOD consists of an active and passive side (see
Figure 3b,c), which mechanically connect the modules once they are equipped with one of them [
11]. As a multifunctional interconnect, the EMI-MOD power and data transfer are ensured after a mechanical connection. For the docking process, by use of the manipulator arm, the EMI-MODs are equipped with LEDs, cameras, and markers to support the above-mentioned visual servoing process. Guiding pins help to assist the mating process. The overall dimensions are 150 × 150 × 60 mm.
The
basic module (see
Figure 3e) serves as a basic structural unit and is a carrier platform for electronic slide-in modules.
The basic module dimensions are 600 × 300 × 180 mm with a weight of 3.3 kg. Larger structures are possible by combining several modules with each other or by scaling one basic module to larger dimensions. The electronic modules, which are to be integrated in the basic module, are available as a modular system in widths of 60 mm, 120 mm, and 180 mm, with a maximum of up to 480 mm. These modules are equipped with an electrical plug-in system, which ensures the power supply and data transfer between the modules and with peripheral electronics. Passive EMI-MODs are fitted on top of the basic module.
Payload modules with dimensions of 150 × 150 × 150 mm combine two EMI-MODs (active and passive) with a standardized payload container (see
Figure 3d). This allows one to easily add new function blocks for system extension and reconfiguration into the toolbox. For the proof-of-concept demonstration, it is envisioned to implement three distinct payload modules, like a camera module, gas sensor module, and battery module. The payload modules can be picked up by the manipulator arm via the EMI-MOD.
The
inter-modular interface (IMI) (see
Figure 3g), with a height of 39 mm and a diameter of 151 mm, allows for a fast manual connection and disconnection between two components. Inside the flanges are freely configurable inlets that contain the components for power and high speed data transmission. The mechanical connection is statically designed for a torsion of 314 Nm. The selected COTS are the rover
Hunter SE with a sensor carrier R&D Kit Pro from Agile X (see
Figure 2), as well as one active and one passive
iSSI® (see
Figure 3h) from iBOSS GmbH.
To integrate the iSSI
® within the mission, the active iSSI
® is attached on an adapter together with the passive EMI-MOD (see
Figure 3h,i). The adapter can be docked on the end effector of the manipulator arm with the passive EMI-MOD.
Another part of the toolbox is a quasi-direct drive motor for space applications, i.e., the
DFKI-X2D joint. The main focus was on development with tests until Technology Readiness Level 5 was achieved, which is the next step toward space maturity [
12].