This section focuses on analysing the traditional way of creating robotic applications, and is split into three parts. The first briefly goes over the state-of-the-art concerning existing robotic middleware and architectures, since these are the core of almost every robotic development endeavor. The second part investigates the big picture, i.e., the steps a robotic developer must usually take, from procuring the robot to the distribution of the final application, where the aforementioned robotic middleware plays a crucial part. Next, the drawbacks and bottlenecks of this procedure are highlighted and the way the proposed R4A approach tackles them is described in the context of citizen development.
2.1. State of the Art Robotic Middleware and Frameworks
The state of practice dictates that robotic software development is performed using robotics middleware. Nevertheless, the first attempt to formalize robotic software development was introduced in the nineties, when several robotic architectures were proposed, which served the task of facilitating the development of robotic systems by imposing beneficial constraints on their design and implementation without being overly restrictive [6
These first approaches followed the classical artificial intelligence paradigm, where the world representation should be as complete and valid as possible, regardless of the robotic actions or tasks, one example of which being the NASREM architecture [7
]. These types of architectures can be described as hierarchical [8
], whose successors were behavior-based (BB)
and reactive architectures [9
]. Of course, hybrid hierarchical/reactive architectures existed as well, and benefited from both approaches, such as AuRA [11
As software architectures became more formal with the use of object-oriented programming (OOP) and software patterns, similar concepts were transferred to the robot development domain. The traditional architectures gave their place to modern robotics middleware, which, first of all, offered several ready-to-use tools and, secondly, promoted the software re-usability concept by referring to behaviors (or code chunks in general) as components [12
]. Some examples of early modular middleworks are Nexus [14
], GenoM [15
] and PredN [16
]. Following the reusable components concept, Nesnas et al. proposed the CLARAty architecture in [17
], which was based on the hybrid reactive and deliberative systems, followed by the well-known robotic middleware called Player, along with its 2D and 3D simulators, Stage and Gazebo [19
]. Of course, middleware with real-time capabilities existed, including Open Robot Control Software (OROCOS) [20
], suitable for motion control, and ASEBA [21
], targeted for microcontrollers incorporation.
Modern robotics frameworks and middleware follow abstract, component-based approaches. Some examples are MARIE [22
], Modular Controller Architecture V2 (MCA2) [23
] OpenRDK [24
], OpenRTM [25
] and SmartSoft, described in [26
]. One of the major breakthroughs in the robotic middleware stage was the release of the Robot Operating System (ROS) in 2009 [27
]. ROS is described as a meta-operating system and was designed to offer hardware abstraction and a standardized module-based development over a structured communication layer. Moving on to more modern concepts, Cognitive Robot Abstract Machine (CRAM) is a software toolbox for the design, implementation and deployment of cognition-enabled autonomous robots, oriented for everyday use [28
], utilizing the KnowRob knowledge processing system [29
]. If modularization is a concern, BRICS is one of the most known meta-frameworks, providing a set of guidelines, meta-models and tools for structuring robotics software via components that were developed using the most known component-based middleware (OROCOS, OpenRTM, ROS, SmartSoft) [30
], adopting the benefits of model-driven engineering approaches, as described in [31
Lately, the appearance of the term “robotic applications” emerged, not meaning the conventional collection of code units, but an actual packaged software that can be downloaded and executed seamlessly in a robot, thus promoting the reusability of such code. The need for robotic applications comes as a result of the inherent complexity of the robotic software development procedure, as explained in the introduction. In [33
], BABEL is proposed, which tackles the multi-domain skillset that robotics require. On the other hand, the RAPP project [35
], developed a new robotic architecture where robotic applications could be created just by installing a robot-specific middleware in each robot and using a robot and cloud API.
Even though it is of great importance to select the proper middleware, it describes only a part of the equation needed to be solved by the developer, especially when he/she is assigned the whole process of setting up the robot, developing the application, testing it and deploying it. Next, an overview of the necessary steps for the developer to take will be presented, where, undoubtedly, an incorrect middleware selection generates ripples of problems in the overall procedure.
2.2. Robotic Applications: The Conventional Way
Robotic system engineering varies greatly. However, it remains an engineering task and, as such, it can be decomposed to distinct, closely related phases. One can identify at least five phases: (a) the setup phase, which includes the procurement and the setup/configuration of the robot; (b) the design phase, where the architecture of the application is designed; (c) the development phase, where the application is developed, usually employing simulators to run component tests; (d) the testing phase, where the code is tested and evaluated on the real hardware; and, finally, (e) the distribution phase, where the application is packaged and distributed for use.
The robotics software development phase usually follows a hybrid process, where functionality is built in an iterative mode, but must be evaluated in a real world stochastic system comprising physical agents (robots) and unexpected situations (environment). It is literally unfeasible to develop software that is correct by design and will be correctly deployed in a robot, regardless of the environmental conditions that appear in the deployment area. As Figure 1
indicates, one of the strategic decisions that the development team must make concerns the testing phase. Some methodologies are proactive and enforce the development of functional software tests before the actual development (Case A), whereas others propose initial development to precede testing (Case C) and some prefer concurrent testing and development processes, such as the agile software development paradigm (Case B).
In this section, a description of the choices and alternative paths a robotics developer is faced with, for all five phases, is presented. The inner structure of these phases is analytically defined by the activity diagrams in Figure 2
, Figure 3
and Figure 4
2.2.1. Setup Phase
Naturally, the journey of the robotic applications developer begins with procuring the robot. The type of robot procurement varies depending on the final application, the path of research or the budget/available funding. The first step to carry out after acquiring the robot is to check if this robot is equipped with a processing unit and if it is modifiable regarding the hardware parts, i.e., if you can attach more sensors or effectors. Similarly, one has to check if in-robot software development and deployment is allowed. There exist cases where even the operating system is proprietary and locked to factory settings, and thus modifications and direct extensions are not allowed.
Concerning the HW part, if the robot does not contain a processing unit, the developer has to attach one and connect the existent sensors and effectors to it. Either way, if the robot allows for HW modifications, the roboticist must choose the suitable sensors and effectors for the application at hand, attach them to the robot body and connect them with the processing unit.
Regarding software development, if no in-robot development is allowed (i.e., the robot is HW-closed), the developer must check if the robot manufacturer provides an external API for controlling the robot. If not, this means that the robot software is closed-source and that its operations are fixed, and thus no application programming is supported. In the case where in-robot or remote (via APIs) development is supported, the roboticist must research if the manufacturer provides access to the sensors and effectors, or if any open-source or proprietary middleware packages exist that would allow him/her to affect the robot behavior (e.g., ROS). If no such tool exists, then unfortunately the developer must build the necessary drivers to access the robotic devices/parts before the development phase, whereas, if middleware packages exist, they must be downloaded and set up in the robot.
As clearly stated, the setup phase in robotics development requires a large amount of effort and multi-domain skills, such as low-level hardware knowledge for constructing the physical part of the robot (base, sensors, actuators, power distribution, etc.) and expertise in software engineering and development. The “conventional way” of the setup phase, regarding robotics application development, is evident in Figure 2
2.2.2. Design Phase
The design phase assumes that the setup phase has been finished, and thus the respective tools (hardware and software) are in place. One important step that all developers must make before the actual development of their application is to design the individual robot software and the overall software system architecture according to the use cases, functional and non-functional requirements. This is an important process, since the designed architecture will allow for the developer to specify required algorithms, multi-threading or asynchronous executors support.
The next task that burdens the developer is the programming language selection. The language decision must be taken under several restrictions and assumptions. Some of the factors affecting the programming language selection are the OS, the selected middleware or the available drivers/APIs, the time criticality of the final application, performance and memory issues and threaded execution or not, as well as the availability of ready-to-use algorithm implementations. For example, if performance/real time issues exist, the selected language must be at a low(er) level in order to capitalize on the low-level calls to the OS. Furthermore, if the selected middleware (or the existing API) provides bindings to, e.g., Python, the developer must make a decision regarding whether this language can be directly utilized or whether wrappers should be created.
After (or during) the language selection process, the developer must pay attention to the selected middleware. If (a) the envisioned architecture requires multiple threads (or processes), (b) the robot CPU supports these requirements and (c) the middleware allows for distributed/modular application creation, then developers must first decide on the deployment strategy, i.e., which operations will exist in which modules. In the case where the CPU is low-end or the application requires more memory than available, the developer should look into the remote access capabilities of the robot, i.e., if remote access is natively supported via being provided by the manufacturer API or the selected middleware. If remote access is indeed supported, the developer may need to offload some components that require high computational or memory resources and distribute them to one or more physical devices/nodes. Nevertheless, this process is quite demanding, as the roboticist should have knowledge of distributed application execution techniques and protocols between heterogeneous devices. Furthermore, communication and message passing between the main controller and the distributed components is almost always dictated by the selected middleware or is custom made, making the remote modules not reusable between robots.
The state of practice assumes that, since the robotics community offers a vast collection of robotic algorithms and modules, the developer should take a look in some of the available repositories in order to not “reinvent the wheel”. After identification of the required modules (if any), the connectivity to the rest of the developed code must be explored. Finally, a message map of the module collaboration must be created. After this step, the developer proceeds with the actual application development.
2.2.3. Development Phase
Since robotics deal with multi-level uncertainty due to the software deployment in a real-life system, it is common to follow an iterative development process, i.e., to create a part of the software, test it, correct it and so on. Apart from bugs in software, the introduced uncertainty may be due to robot motion failures and sensory malfunctions or sensors’ deviation from the nominal measures, a fact that enforces the employment of probabilistic theory in almost all algorithmic domains of robotics (SLAM, navigation, path planning, etc.)
After each development iteration, the roboticist must check and evaluate the functional behavior of each software component. When a simulator is available and the code can be deployed on it, the developer must install and setup this tool, perform tests and then reiterate the development procedure. Runtime tests must also be performed on the real robot and in the context of a real environment. If remote execution is supported by the underlying middleware, the connectivity to the robot must be established prior to the deployment of the application. If not, the relevant binaries must be manually transferred to the physical robot and deployed. If the software performance is satisfactory, the developer proceeds to the next phase, which is testing or distribution (according to the favorite development strategy).
2.2.4. Testing Phase
Initially, the developer must choose appropriate testing tools based on (a) the under testing functionality or component and (b) the selected programming language. The selection of the proper testing tools must be made early, especially if an agile software development methodology is applied and the testing and development processes run concurrently. Concerning the actual testing, the most common types of tests are unit, functional, integration and system tests. Unit tests can usually be deployed without a real robot or even a simulator, since they test specific parts of the software; nevertheless, one can deploy them in a simulator or on the real robot as well. As far as functional and integration tests are concerned, the robot must be present, either in its physical form by employing a simulator, or by creating mock endpoints for sensor/effector API calls. System tests are performed on a complete, integrated system in order to evaluate compliance with the initial requirements, and thus these must be deployed in the real robot under realistic environmental conditions. One common practice in software development is continuous integration (CI), where a remote framework is set up to monitor code changes and execute all provided tests in order to offer a continuous status report on whether the software behaves as it should. If a CI is not present, the tests deployment must be manually performed.
2.2.5. Distribution Phase
When the developer is satisfied with the quality of the code, the created software matches the requirements of the setup phase and the produced code solves the problem at hand, the distribution phase follows, provided the developer wants to share.
Distribution methods highly depend on the runtime platform, both at the hardware and operating system layers, and the software license governing the use and redistribution of the software. In the case of closed-source distribution, the following (mainstream) methods can be used:
Make the pre-compiled binaries available to the web via a public link;
Package and upload to OS-specific upstream repositories (e.g., Debian package);
Package and upload to specialized repositories of robotics software, according to the selected middleware (for example, ROS upstream repositories).
On the other hand, open-source software can also be hosted by version control systems, such as Git, Subversion, etc. In all the above options, the developer must create the necessary documentation and tutorials that explain, in detail, how to download, setup, install and deploy the software. If the software is a generic algorithm that can be applied to a number of robots (e.g., a SLAM algorithm provided as a ROS package), it is helpful to create tutorials for different robots, aiming at assisting a beginner roboticist.
2.3. Identifying the Limitations for Citizen-Developers
As evident from the above description, the development process includes a number of choices to be made, where most of which are only meaningful to an experienced developer. Wrong choices can lead to several issues, which may be critical enough to force the re-design and re-implementation of the whole system.
2.3.1. Resource-Related Inability to Meet the Requirements
In the first step (setup/design), it is important to be perfectly clear what the robot offers for control and data acquisition purposes, as well as the way it provides it. In case the developer has not performed thorough market research, the selected robot may lack functionalities or resources crucial to the final application.
2.3.2. Middleware-Related Inability to Meet the Requirements
During the software development phase, the developer must choose one of the decades of available robotics middleware and possess the required knowledge of the robot’s OS in order to correctly perform the configuration. As understandable, problems may arise if the selected middleware does not inherently offer requirements-related functionality (e.g., ROS employment for controlling a real-time system is not a suitable selection).
2.3.3. Portability Problems to Other Robots
Even if the previous choices were wisely made, it is quite certain that the produced software will not be able to be deployed on another robot unless the developer has taken into account cross-robot execution support.
2.3.4. Complexity Due to Distributed Application Deployment
If the robotic embedded hardware platform is not powerful enough (resource-wise), it is quite hard to manually develop the infrastructure and to support a distributed or even remote execution of the code.
2.3.5. Physical Damage Due to Insufficient Testing
During the testing and deployment phases, if the developer does not follow the standard testing procedures and these are performed on the physical robot platform and in real environments, it can even lead to physical damage of the robot (e.g., during the development of an obstacle avoidance algorithm).
2.3.6. Erroneous 3rd Party Setup/Execution
Finally, the distribution phase is quite demanding, since a lot of time must be spent writing error-free instructions for the code retrieval, installation, configuration and deployment.
2.4. Leveraging the Robotic Software Development Problems
This work directly aims towards establishing a rapid application development methodology in robotics for end-users to be able to integrate robotics in their development process and products. Currently, in robotics, in order to reach the application development phase, excessive effort and multi-domain knowledge are required in order to develop the control and monitoring software stack of a robot. By performing high-level abstractions, the undesired effect of restricting experienced developers arises. For this reason, R4A supports the seamless integration of ROS components (since ROS has become the standard as far as robotics middleware is concerned). This way, anyone can include low-level implementations in robot-agnostic applications, or can even utilize algorithms from the ROS ecosystem. Through the R4A architectural approach, one may ensure standardizing the robotic applications development in the context of “robotics for citizen developers”.
As far as the standardization of the robotic software development is concerned, the robotic development process includes numerous different decisions to be made based on the work-flows used in the past, the strategy of the involved team or even knowledge retrieved from the Internet. Using the proposed architecture, a new process is introduced in the development cycle, since the steps from the robot acquisition to the application distribution are clearly stated.
For citizen developers, although most of the traditional robotics middleware offer tools, ready-to-use libraries and formalisms to support the development of robotic functionality for complex application domains, things are far from easy. Such frameworks aspire to be flexible towards the user needs; this practically means that they intend to solve the problem optimally, rather than easily. Additionally, they do not support reusability, i.e., to easily execute applications (supported by the core functionalities) in different robotic platforms. The correct configuration sometimes lies within the authority of the build-in software stack and cannot be easily changed or even changed at all. The current work aspires to tackle some of the aforementioned mainstream problems.
2.4.1. Installation and Configuration
Concerning the first phase, during which, the developer must investigate the ways to control the robot and fetch data from its sensors, the creation of a robot-aware module is proposed, which will be easily installed and deployed in a robot and will uptake the role of the robotics middleware. Since this will be a per-robot software module, the burden of complex installations and configurations is eliminated.
2.4.2. Remote Data Acquisition and Control of On-Robot Effectors
The R4A approach provides a robot-agnostic API, meaning that the developer will be able to create applications without having perfect knowledge of the robot sensors and effectors, or even the exact morphology of the robot. The creation of such an API is possible, since the information available and the control of different robots has several common elements. The server-side robot API provides web services using a resource-oriented architecture and can enable both the local and remote execution of robotic applications and behaviors. This way, developers will be able to create complex applications, even for low-end robots.
2.4.3. Simplified Testing Procedures
The middleware functionalities should be tested beforehand by robotic experts, and thus only functional testing is required to be performed by the developer. Integration testing is not required due to the utilization of the abstract robotics middleware.
2.4.4. Seamless Portability/Distribution
Since the applications will be robot-agnostic, the procedure to be executed in a different (but similar) robot will simply require the installation of the specialized middleware and the deployment of the application.