Next Article in Journal
Control of Power Electronics through a Photonic Bus: Feasibility and Prospects
Next Article in Special Issue
Activity Classification Feasibility Using Wearables: Considerations for Hip Fracture
Previous Article in Journal / Special Issue
Design and Analysis of Adaptive Hierarchical Low-Power Long-Range Networks
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JSAN
Volume 7
Issue 4
10.3390/jsan7040052
jsan-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Networking of Multi-Robot Systems: Architectures and Requirements

by
Imad Jawhar
1,*,
Nader Mohamed
2,
Jie Wu
3,* and
Jameela Al-Jaroodi
4
1
Faculty of Engineering, Al Maaref University, Beirut 1600, Lebanon
2
Middleware Technologies Lab., Pittsburgh, PA 15057, USA
3
College of Science and Technology, Temple University, PA 19121, USA
4
Department of Engineering, Robert Morris University, Moon Township, PA 15108, USA
*
Authors to whom correspondence should be addressed.
J. Sens. Actuator Netw. 2018, 7(4), 52; https://doi.org/10.3390/jsan7040052
Submission received: 12 October 2018 / Revised: 16 November 2018 / Accepted: 19 November 2018 / Published: 30 November 2018
(This article belongs to the Special Issue Sensor and Actuator Networks: Feature Papers)
Browse Figures
Versions Notes

Abstract

:
A large number of advancements have taken place in microprocessor-based systems leading to significantly more processing, memory, storage, sensing, actuating, recognition, controlling and communication capabilities. Robotics is one of the areas that have benefited a lot from these advancements. Many important and useful applications for single-robot and multi-robot systems (MRS) have emerged. Such applications include search and rescue, detection of forest fires, mining, construction, disaster management, and many more. MRS systems greatly enhance the capabilities and effectiveness of today’s robots. They extend the robotic system capabilities by increasing the ability to perform more complex tasks and allow performance of inherently distributed ones. In addition, they increase parallelism, enhance robustness, and improve system reliability. However, to perform their tasks in an effective manner, communication between the individual robots becomes an essential component. In this paper, we discuss the various types and architectures of MRS systems and focus on the networking issues, and services that are required to enable MRS systems to be more efficient in performing their roles in their respective applications. We also identify the similarities and differences between mobile ad hoc networks (MANETs) and MRS systems, analyze robot-to-robot (R2R) and robot-to-infrastructure (R2I) communication links, and identify the protocols that can be used at the various levels in the MRS hierarchy.

1. Introduction

The recent technological advances in robotics has opened the door for many useful automation applications in diverse domains. These advances include major enhancements in sensing, actuating, processing, analyzing, communications, recognition, leaning, perception, and controlling capabilities. With such advancements, many applications can be enhanced with better productivity, accuracy, quality, reliability, cost-effectiveness, flexibility, and safety using robots or robotic systems. For example, these capabilities facilitated the development of advanced industrial robot systems that automate and enhance manufacturing processes to produce more products with better accuracy and quality in more cost-effective manners. They also allowed for the development of farm robots to handles agriculture processes and increase productivity, thus reducing the costs of farming and increasing accuracy in handling crops. Furthermore, they allow to develop maintenance robots to perform inspection and repair tasks in difficult and dangerous places such as in underwater facilities and infrastructures spanning rough and inaccessible terrains. Moreover, they allow to develop security robots that support the monitoring of important infrastructures such as oil, gas, and water pipelines and power stations. All these and many other applications will emerge and become more common in the near future as the driving technologies in robotics and supporting systems mature.
While some applications require the use of a single robot to effectively automate and enhance a process, there are others that require using multiple robots working together to achieve certain objectives in more cost-effective or/and efficient manners. Multiple robots used for any of these applications can be combined in what is referred to as a Multi-Robot System (MRS). It is worth noting that some researchers use the term Multi-Robot Network (MRN). Although both terms refer to the same systems, we will use the term MRS subsequently in this paper. Any MRS has several robots working together to achieve a certain application objective. Using an MRS over a single robot provides many benefits [1,2]:
  • Multiple robots can concurrently work on the task to achieve it faster.
  • Robots can be heterogenous in their capabilities to provide a cost-effective solution to achieve a task where each robot handles specific components of the task matching its capabilities.
  • Multiple robots can effectively deal with a task that is inherently distributed over a wide area.
  • Using multiple robots for achieving a task provides fault tolerance as the presence of multiple robots capable of similar processes can be used to compensate when any of them fails.
There are many research efforts in investigating different issues in MRS and providing solutions for these issues. The issues include task allocation [3], distributed intelligence [4], learning [5,6], coordination [7], coalition formation [8], motion planning [9] and middleware support [10]. In addition, there are some research efforts dedicated for developing application-specific solutions and algorithms for MRS such as patrolling algorithms [11], map merging [12], box-pushing [13], and flocking [14]. However, there is little effort in investigating the communication and networking aspects of MRS. Yet, communication and networks represent the main enabling technology for MRS applications. Robots in an MSR are connected with a network and the performance of this network has direct impact on most other solutions in MRS as well as the final performance of the applications. Some of the available work in this area focus on studying some theoretical aspects of MRS such as communication complexity [15] and localization [16]. In addition, there are some efforts to study specific communication issues for specific types such as small, low-power, low-cost MRS [17] and UAVs [18]. In this paper, we investigate the communication issues and networking models for MRS. We also study the communication requirements of different services in MRS. This investigation is motivated by the recent developments in communication, networking, and increasing demand on applications using MRS in real-life.
The rest of the paper is organized as follows. Section 2 presents the types of MRS systems and their applications. Section 3 identifies the different types of MRS architectures. Section 4 offers an overview of the related communication issues and requirements. Section 5 provides a discussion of cloud and fog-based MRS systems. Finally, Section 6 concludes the paper.

2. Types of MRS Systems and Their Applications

2.1. Types of MRS Systems

In general, there are two types of MRS systems [1].

2.1.1. Collective Swarm Systems

This type is characterized by having a typically large number of mobile robots which execute their own tasks based on local control laws leading to coherent team behavior. They require minimal communication with other robots.

2.1.2. Intentionally Cooperative Systems

This type is characterized by robots that have a knowledge of the presence, state, actions, and capabilities of other robots in the team. They work together to accomplish the same objective. This type of robots is further divided into two categories: (1) Weakly cooperative and (2) Strongly cooperative. This classification is based on the extent to which the actions of each robot is affected by the state and behavior of other robots in the team.
  • Strongly cooperative: In this class of MRS systems, the robots act cooperatively to achieve a common goal. Consequently, this kind of close coordination require appropriate communication and synchronization which typically has more stringent quality of service (QoS) requirements such as bandwidth, and delay.
  • Weakly cooperative: In this class of MRS systems, the robots divide and coordinate the tasks that need to be done among themselves. Afterwards, each robot proceeds to achieve its task with a form of operational independence. In this case, the supporting communication protocols and corresponding QoS requirements are more relaxed.
Furthermore, intentionally cooperative MRS systems can have heterogeneous robot members, which vary in sensing and acting capabilities. Consequently, they are different than the collective swarm systems, due to the fact that the robots belonging to a certain team are no longer interchangeable.

2.2. MRS Applications

There are several applications that can be improved using MRS. Some of these applications are search and rescue [19,20], detection of forest fires [21], hazardous waste removal [22], farm operations [23], mining [24], constructions [25], disaster management [26], security applications [26,27], warehouse management [28], moving containers within harbors and airports [29], and gaming and entertainment such as soccer [30].

3. MRS Architectures

Various major architectures can be identified for MRS systems [1]. Such architectures significantly affect the robustness, reliability, and scalability of the system, and are dictated by the strategy used to make decisions, manage the interactions between the robots, and generate the group behavior of the team.
  • Centralized: In this category a single point of control manages the behavior of all the robots in the team [31]. Such architecture suffers from the single point of failure problem, which can reduce its reliability. Also, the scalability is diminished. This is because the central controller must be constantly aware of the state of the all the team members which triggers the exchange of numerous messages in addition to the control messages which must also be sent back to the individual robots to control their actions.
  • Hierarchical: In this category, the robots are organized in a command and control hierarchy similar to that in the military. Specifically, in this strategy, a robot controls a group of other robots. Each of those robots in turn controls a group of other robots. This pattern can continue for several levels down the hierarchy depending on the size of the network. This approach, it is highly scalable and can be appropriate for some applications with a large number of robots. However, it has reduced reliability due to the considerable vulnerability in handling failures of robots at the higher levels in the hierarchy.
  • Decentralized: This is the most common category for MRS systems. In this case, robots take actions based on their own local view following certain strategic guidelines and goals for the team. This model is characterized by its robustness and ability to adjust to failures, since no centralized control is used. On the other hand, it is a challenge to keep the synchronization and coherency among the robots. In addition, it is not trivial to coordinate actions when mission objectives change.
  • Hybrid: This approach combines a local decentralized control, which provides robustness with hierarchical control to achieve global synchronization and coordination of actions, goals, and tasks. This hybrid strategy is used in many MRS systems, which need to have scalability due to the large size of the network as well as an ability to take quick decisions on the local level to achieve better performance and quicker reaction to local events and failures.

4. Communication Issues and Requirements for MRS Systems

Two types of communication links can be identified for MRS systems to be able to accomplish their various tasks and services: Robot-to-robot (R2R) and robot-to-infrastructure (R2I). This is illustrated in Figure 1.

4.1. MANETs and MRS

Since robots are mobile during a significant portion of their operational time, they constitute a specific type of mobile ad hoc networks (MANETs). Certain protocols that are appropriate for MANETs can be used at the various layers of the open system architecture (OSI) model. These layers include the physical, data link, network, transport, and application layers (The session and presentation layers are commonly merged with the application layer).
At the physical and data link layers, which constitute the underlying network, MRS communication can use the IEEE 802.11 (Wi-Fi) protocol. It has a line-of-sight range of several hundred meters. The latest version is the IEEE 802.11ac protocol, which has higher data rates (up to a theoretical rate of 3.4 Gbps) that can support a wider range of data traffic. It uses the carrier sense multiple access with collision avoidance (CSMA/CA) protocol at the medium access control (MAC) layer as well as orthogonal frequency division multiplexing (OFDM) with multiple-input and multiple-output (MIMO) technologies at the physical layer. MRS communication can involve both best effort (BE) as well as QoS guaranteed traffic. For the earlier type of traffic, which is delay-tolerant, the distributed coordination function (DCF) mode can be used, which involves a contention-based access strategy. For the latter type of traffic, which might be delay, jitter, and bandwidth sensitive, such as audio, video, multimedia, and real-time, the point coordination function (PCF) mode can be used, which involves a contention-less access strategy using guaranteed time slots (GTSs). The super frame in the IEEE 802.11 protocol allows both types of accesses in its corresponding two phases.
Since the network layer is responsible for end-to-end routing from the source node to the destination, we can take advantage of the large amount of research that is done for MANET routing protocols, which can be used and adopted for MRS systems. Such protocols can be classified into the following categories:
  • Static routing: In this class of protocols, the route is pre-programmed in to the routing tables of the individual nodes before the mission. Although this can be useful for some missions with relatively fixed and predetermined topology, it is inflexible and very limiting for MRS systems, which are characterized by higher mobility and variable topology.
  • Proactive routing: In this class of protocols, a routing table is constructed and periodically maintained by the nodes even before transmission requests arrive from the upper layers to the network layer. This strategy increases the overhead of route discovery and maintenance messages which reduces efficiency in cases where the node-to-node traffic is limited and decreases the scalability of the corresponding MRS network. However, it can be suitable in cases where the application cannot tolerate large and unpredictable route discovery delays such as MRS networks with high real-time coordination involving most of the nodes. The destination-sequenced distance-vector (DSDV) protocol is an example of such protocols.
  • Reactive routing: In this case, an end-to-end path between the source and the destination is only discovered on-demand when it is needed. Once the route is discovered the routing table at the source node is updated with the corresponding entry. All subsequent transmissions from that node to the destination use the discovered path as long as the related topology and relative locations of the intermediate nodes are not changed in a way that leads to the breaking of one or more intermediate links. If the path is no longer valid, a new path discovery process is initiated. This routing strategy reduces the control message overhead, since the routes are only discovered when they are needed. It is appropriate for MRS networks with a relatively larger number of nodes and lower traffic which is limited to the occurrence of certain events with a selected number of nodes involved such as search and rescue, and environmental monitoring applications. Dynamic source routing (DSR), ad hoc on-demand distance vector (AODV), and temporally ordered routing algorithm (TORA) are examples of such protocols.
  • Hybrid routing: In this case, both proactive and reactive routing strategies are used in the same MRS network. This type of routing aims at benefiting from the advantages of both strategies by applying them under different conditions. For example, the network can be divided into multiple geographic clusters with selected cluster head (CH) nodes in each cluster. Consequently, intra-cluster traffic, which consists of data exchanges among nodes in the same cluster uses a proactive routing strategy since the nodes are likely to have more frequent exchanges due to closer cooperation and tighter coordination. The inter-cluster traffic, which consists of data exchanges among nodes in different clusters uses a reactive routing strategy, which routes the data among the CHs in different clusters. Traffic that is exchanged among nodes in different clusters is routed to the CH of the source cluster, which transmits the data to the CH in the destination cluster. The latter subsequently routes the data to the destination node in that cluster. This approach provides a scalability advantage and is appropriate for larger MRS networks with clusters of robots responsible for closely coordinated tasks in the same geographic area (or cluster) but need to communicate information to robots in other geographic areas (or clusters) less frequently.
  • Location/Geographic routing: In this type of routing, each node must be equipped with GPS circuitry or must be provided with the position information of other nodes in the network from a location service. This comes at the expense of increased node cost. Data is routed from the source to the destination using various routing strategies which are intended to keep moving the data closer to the geographic location of the destination until the latter is reached. This type of routing has the advantage that each node does not need to keep track of the topology of the network. It only needs to determine the next hop that gets the data closer to the destination. The routing overhead is reduced since the routing decision is based on local information and the routing process typically incurs lower delays. This type of routing can be useful for MRS networks stretching over larger geographic areas and the nodes are highly mobile leading to constant topology changes. Protocols in this category include Location-Aided Routing in mobile ad hoc networks (LAR) [32], Energy Efficient Location-Aided Routing (EELAR) [33], greedy location-aided routing protocol (GLAR) [34], Location-Aided Energy Efficient Routing (LAEER) [35], and Location-Based Efficient Routing Protocol (ALERT) [36].
  • Hierarchical protocols: In this case, a multi-level hierarchy is adopted. Consequently, communication between the lower level nodes is done by passing the data to the upper levels. For example, a three-level hierarchy can be used where the lower level robots play a sensing/acting role. They are identified as basic robots (BR). The robots at the second level play a data relay role, where the data between various clusters of lower level robots is relayed among the data relay robots (DRR). The robots at the third level are identified as data dissemination robots (DDR), and are responsible for delivering the data collected by the DRRs to the control center (CC). This type of strategy provides higher scalability at the cost increased complexity, and cost. However, for MRS networks with a large number of small robots (hundreds or thousands) covering a wide geographic area, this hierarchical model would be appropriate. Hierarchical routing protocols include Low-Energy Adaptive Clustering (LEACH), Power-Efficient GAthering in Sensor Information Systems (PEGASIS), and AdaPtive Threshold sensitive Energy Efficient sensor Network protocol (APTEEN) [37,38,39].

4.2. MRS Networking Protocol Issues and Requirements

Choosing the proper routing protocol is an important decision, which affects the performance of the network. It is critical to choose one that is appropriate for the type of MRS system and the corresponding application that is being supported. The following issues and requirements are essential to be considered resulting in efficient network operation.
  • Number of robots: Some networking protocols operate efficiently only when the number of nodes in the network is small. Such protocols might not be practical for large networks. This can be the result of a high number of control message exchanges that are not localized to the event area for example.
  • GPS capability: Depending on the routing protocol, some routing strategies might assume that each node knows its location as well as the location of its neighbors. This leads to an increase in node cost and can be detrimental for MRS systems where a large number or inexpensive robots is desirable.
  • On-board processing: Some routing protocols require a large amount of processing, which might not be available in some or all the nodes in an MRS networks. For such networks, it is essential to choose a simple routing protocol that can make its routing decisions without complicated logic that takes too many parameters into consideration.
  • On-board memory: The amount of memory required to store routing table information as well as other routing-related parameters varies widely between different routing protocols. For example, link-state protocols require the topology of the entire network to be stored by each node, which significantly increases the amount of memory needed for larger networks. This is not the case for distance-vector protocols. Consequently, for large MRS networks with small inexpensive robots, it is important to choose protocols, which do not require a lot of memory. This requirement is not so important for smaller networks with more capable larger robots with considerably larger memory resources.
  • Energy: Robots come in various sizes and shapes. They range from tiny robots that are smaller than the human hand, or even microscopic to ones, to ones that are as big as a large-size UAV. However, in most cases, they draw their energy from a battery, which has a limited capacity. Consequently, the networking protocols must have the reduction in energy consumption as a major goal. This can be done using numerous strategies. Such strategies heavily depend on the application that is involved, the level of responsiveness and readiness of the robots, as well as the nature of the tasks that they performed.
  • Network throughput: Different applications and the related services and tasks performed by the robots and the MRS network require varied amounts of communication bandwidth. Depending on the network conditions and the amount of traffic that is being exchanged, there might be dropped messages due to congestion, collisions, interference, and delay. As a result, the throughput of the network might vary considerably leading to problems with applications that require higher data rates and can tolerate only a limited amount of delay. Consequently, the network protocol needs to ensure that the specific throughput that is needed by the applications is satisfied by the underlying network.
  • Co-located networking protocols: In addition to the routing protocol that is used to route messages between robots (R2R link) in an MRS system at the network layer, it is highly likely to have other wireless protocols that are used by R2I links or other wireless networks that are in the same area. Consequently, it is important to take that into consideration when choosing the right protocol to reduce the possibility of interference when the protocols operate in the same frequency range. This is especially important in MRS systems that perform critical missions where errors can cause life-threatening or catastrophic results.
  • Connection to backbone: In the cases where the MRS network needs to be connected to a backbone network or to the Internet, there is a need to map the QoS and other networking parameters of the R2R and R2I traffic from the header of the MRS packet to the corresponding parameters in the headers of the upper layers in the backbone network. This should be done in a way that ensures seamless integration of the two systems, which does not lead to appreciable changes in the QoS of the associated MRS traffic.
  • Mobility: Nodes in an MRS network are typically mobile. However, the mobility degree and pattern can vary widely between different applications. For example, MRS networks with nodes consisting of UAVs are expected to move at considerably higher speeds than nodes in an MRS with nodes consisting of robots moving on rough terrain in a search-and-rescue mission. Consequently, the routing protocol that is chosen for the MRS network must be characterized by higher efficiency for the mobility degree and pattern of the particular MRS network.
  • Handoff and roaming: MRS networks with a very large number of nodes (hundreds or thousands) typically need to have multiple layers of hierarchy as indicated earlier. Robots in a particular cluster usually transmit their data to an elected CH or a gateway node that provides connection to the rest of the robots in the MRS or a backbone network. However, when a particular robot goes out of range of its designated or current gateway and into the range of another gateway, a handoff process needs to take place. When robots require changing their connection from a particular network to another one due to geographic movement a roaming mechanism must take place (for example, the MRS might be using cellular connections). Such considerations are important to keep in mind at the network design and configuration phases.
  • Security: Depending on the nature, confidentially, and criticality of the MRS application, secure communication might be an essential requirement to ensure its success. Certain MRS networks that perform military, or sensitive commercial services might be subject to many kinds of attacks. The attacks can be internal or external to the network. Consequently, the communication protocol that is chosen must have security provisioning to ensure protection against possible attacks. Such attacks can be passive or active. Passive attacks include eavesdropping, and traffic analysis. Active attacks include masquerade, replay, modification of message content, and denial or service (DoS).
  • Reliability: To varying degrees of importance, reliability of the communication process is an important aspect of certain MRS networks. For certain types of applications, R2R and R2I communication cannot tolerate interruptions, which might lead to mission disruption, or failure, which in turn can result in high financial losses or cause life-threatening conditions. Subsequently, the MRS network protocol need to include reliability features such as fail-safe provisioning, redundancy of critical communication components, or designated nodes to replace other communication-critical nodes in case of malfunctioning or failure.

4.3. Robot-to-Infrastructure (R2I) Communication

MRS networks very often need to exchange information with infrastructure networks or the Internet. Consequently, one of the important components in the communication architecture of MRS systems is the R2I link. In this case, one of the robots that are connected to the others through R2R links can be equipped with additional resources, which enable it to play the role of a gateway node provide connectivity with the backbone network and the Internet. The link from the gateway robot to the infrastructure network access point (R2I link) can use longer range access protocols such as IEEE 802.16 (WiMAX), cellular, or satellite depending on communication services available in the geographic area. This gateway node must be able to map the networking parameters associated with the data traffic in the data link layer header to the corresponding data link layer header in the infrastructure network.

4.4. Communication Links in MRS Systems

Figure 2 shows the R2R and R2I communication links along with the different networking protocols that may be chosen for each one. The various networking protocols that may be selected by the network designers are shown in Table 1 along with the corresponding characteristics and specifications for the physical layer, data link layer, data rate, transmission range, and the appropriate link for each one [40,41,42].

4.5. The Networking Layers in the Nodes of the MRS System

The networking layers at the nodes using R2R and R2I communication are shown in Figure 3 and Figure 4 respectively. In Figure 3, the two nodes are either two BR, or a basic robot and a gateway robot. In Figure 4, the nodes consist of a gateway robot on one side and an access point (AP) to the infrastructure (or backbone) network and/or the Internet on the other side. In the first case (R2R communication), the shorter to medium range wireless protocols can be used. Such protocols include IEEE 802.15.4 (Zigbee), IEEE 802.11 a/b/g/n/ac (Wi-Fi), and IEEE 802.15.1 (Bluetooth). The first two cover the physical and data link layers, and Bluetooth covers the same layers as well as the network and transport layers. These protocols are also characterized with relatively lower energy consumption. In the second case (R2I communication), the medium to longer range wireless protocols can be used. Such protocols include IEEE 802.11 a/b/g/n/ac (Wi-Fi), IEEE 802.16 (WiMAX), Cellular, and Satellite. Above the underlying network protocols resides the Internet Protocol (IP) protocol at the network layer, the Transmission Control Protocol (TCP), User Datagram Protocol (UPD), or Stream Control Transmission Protocol (SCTP) at the transport layer. These protocols are followed by the middleware layer, and the appropriate client and server protocols at the application layer.

4.6. MRS Middleware Layer Functions, Services, and Requirements

The middleware layer can be used to provide important services and functions between the lower layers and the application layer in MRS system architectures. These services and functions need to satisfy the requirements for different types of tasks and services that can be considered in MRS systems. The following are examples of such services:
  • R2R communication.
  • Robots acting as relay nodes.
  • Robots acting as gateway nodes (R2I links).
  • Robots assisting in sensing operations.
  • Robot providing data storage services.
  • Robots providing processing services.
  • Implementation of distributed or centralized control of robot teams.
Additional ones can be added depending on the particular MRS application that is being considered.

4.7. Types of Traffic Generated by MRS Systems and Associated Requirements

Depending on the application involved, the QoS requirements of the data traffic generated and communicated by MRS systems can vary in significant ways. Such requirements include bandwidth, delay, and delay jitter in addition to other parameters. The protocols used by the R2R and R2I links must support these requirements to ensure timely and effective communication leading to proper operation and completion of the tasks that are performed by such systems.
For example, MRS teams working on a common task usually require real-time communication with low delay tolerance. On the other hand, MRS teams that are responsible for data collection from wireless sensor networks (WSNs) or taking audio/video information for later analysis need networking protocols with high data rates while tolerating longer end-to-end delays.

4.8. Robot Operating System Networking Services

ROS (Robot Operating System) is a well-known open-source development environment for robotics applications [43]. The ROS environment provides abstractions and visualization for robotics hardware, devices, and motors. This enables ROS to be used for various types and brands of robotics. In addition, it offers many libraries and tools that simplify the development process of robotic applications such as message-passing among processes and package management. A graph architecture is used for executing groups of ROS-developed processes. These processes are executed in nodes that can deal with different types of messages such as communication messages, sensor messages, control messages, state messages, and actuator messages. Some MRSs applications and solutions were developed with ROS. One example is WAMbot, which was developed to use visual object recognition to navigate, discover and map a large urban environment [44]. Another example is a ROS-based multi-robot architecture for ambient assisted living [45]. Furthermore, a collision avoidance technique with localization uncertainty for MRSs was developed with ROS in [46]. The main advantage of using ROS for developing these solutions is using the various available ROS tools and libraries to improve and shorten the development processes. However, these systems, applications, and solutions were developed using some available libraries such as message passing, these libraries are not fully optimized to deal with different networking architectures and requirements in MRSs. More work is needed to create more libraries to provide advanced and optimized communication services that can deal with different networking architectures and requirements.

5. Networking for Cloud and Fog-Based Multi-Robot Systems

In this section, we will discuss two important infrastructures, cloud computing and fog computing, that can provide advanced support to both individual robots and MRS. As part of their support, they need specific network capabilities, which could raise various issues. We will discuss the network issues and requirements pertaining to cloud and fog computing use.

5.1. Cloud-Based Multi-Robot Systems

Cloud computing could be incorporated as an important component of robotics applications as it can provide many powerful and advanced services needed to effectively operate these applications [47,48,49]. Cloud computing offer powerful processing capabilities, large and scalable data storage, and advanced software services that can be used to build different support services to be used in diverse robots and MRS applications. Cloud computing can be used as the main control and management platform to execute MRSs. Different robots, sensors and actuators of an application can be connected to cloud computing services to collect, process, store the robots and sensors’ data and perform management tasks for different robots and MRS applications [50]. As the collected data from MRS and other related systems can become big data, cloud computing can be used to provide the necessary powerful platforms for storing and processing this big data to enhance operations and planning of the robots and MRS applications.
The communication between robots of MRS applications and cloud computing involve different requirements to smoothly support these applications. These requirements should be supported by the network architectures deployed in the MRS. Cloud robotics and MRS applications rely on the integration between different robots on one side and the cloud on the other and cannot perform well unless there is a well-designed network that provides efficient and reliable communication services connecting both sides. Another issue that arises when using cloud computing for MRS applications is that the cloud services are either offered at a centralized location or across multiple distributed platforms in various locations. The distributed cloud computing approach can provide better quality and reliability support for different applications [51]. However, there is usually a need to provide good communication links among the distributed cloud computing facilities in these distributed locations. Another issue arising when using the cloud is the reliability and performance of the networks connecting all components on both sides [52]. With the Internet in the mix, there are problems with delays, lost packets, and unstable connections. Careful planning and management of network resources and communication models in addition to the design and architecture of the MRS application is necessary to account for these issues. Some of these issues may be resolved by the design efforts, yet there are some unavoidable aspects such as the transmission delays.
One emerging networking technology that can provide some solutions for networking challenges in cloud-based MRS is software-defined networking (SDN) [53]. SDN offers some features for configuring cloud-based MRS to support different applications. These include improving QoS, reliability, latency, scalability, and security between MRS and the cloud computing components. While there are some efforts in investigating this approach for supporting other applications such as smart cities [54], there is very limited work to provide similar customized solutions for cloud-based MRS. Therefore, there is plenty of room for developing more advanced management and networking mechanisms in SDN for efficient, reliable, and secure network configurations in cloud-based MRS.

5.2. Fog-Based Multi-Robot Systems

While cloud computing can provide advanced and powerful services for MRS applications, it usually cannot provide provisions for distributed MRS applications that need real-time, mobility, low-latency, data streaming, synchronization, coordination, and interaction support services [55]. This is mainly due to the transmission delays imposed by the large distances to be covered between the robots and the cloud platforms in addition to the usual issues of jitter, packet loss, and broken links that may occur across the Internet connections used. In addition, it is difficult for cloud computing to manage and deal with a large number of heterogenous robots, sensors, actuators, and other devices distributed over a large-area. Fog computing was lately introduced to offer more localized, low-latency, and mobility support services. Fog computing allows moving some functionalities from the cloud to compute nodes situated closer to the devices such as the robots [56]. This approach aims to enable different Internet of Things (IoT) applications over distributed fog nodes that provide localized services to support these IoT applications. For MRS applications, fog computing can complement cloud computing to support MRS applications [57]. While cloud computing can provide powerful and scalable services for MRS applications, fog computing can provide more localized, fast-response, mobility, and data streaming services for these applications. The integration between the MRS, fog computing, and cloud computing, as shown in Figure 5, can provide a powerful platform to support different MRS applications by taking advantage of the best features on each platform. Yet this platform becomes more complex and highly distributed, thus creating more issues for the supporting communication infrastructure that needs to keep up with the imposed demands on the network services. This leads to the need for well-designed networking and communication support to efficiently and reliably handle the communication between all components. Several aspects in the network need to be addressed to create efficient communication infrastructure and minimize network problems across all platforms. This also includes introducing strong network security support to avoid any threats and vulnerability possibilities in the integration and in supporting MRS applications.

6. Conclusions

In order enable MRS systems to achieve their potential in their numerous applications, R2R and R2I communication becomes an essential component that allows for proper synchronization, task allocation, and active cooperation. In this paper, we discussed the different types and networking architectures of MRS systems. We presented the various issues, requirements, and services that need to be addressed to allow MRS systems to perform their respective tasks. Furthermore, we identified the communication protocols that can be used at the various levels in the R2R and R2I networking stack to satisfy the requirements of the corresponding application and their traffic requirements. We also discussed networking considerations and issues for cloud and fog-based MRS systems. Efficient and seamless communication remains an essential part of the MRS system architecture. A good amount of research is still needed around network protocol scalability, energy efficiency, reliability, security, and mobility support leading to the effective design, development, and deployment of MRS systems to improve many aspects of today’s society.

Author Contributions

I.J. wrote the main sections of the paper. N.M. contributed to the sections on robot operating system networking services, and the networking for cloud and fog-based MRS systems. J.W. contributed to the introduction and communication in mobile ad hoc networking systems. J.A.-J. contributed to the sections on the MRS middleware layer and the networking for cloud and fog-based MRS systems. All authors read, and approved the final manuscript, and contributed to various sections in the paper according to their background.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Parker, L.E. Multiple mobile robot systems. In Springer Handbook of Robotics; Springer: Berlin, Germany, 2008; pp. 921–941. [Google Scholar]
  2. Zlot, R.; Stentz, A.; Dias, M.B.; Thayer, S. Multi-robot exploration controlled by a market economy. In Proceedings of the IEEE International Conference on Robotics and Automation, ICRA’02, Washington, DC, USA, 11–15 May 2002; Volume 3, pp. 3016–3023. [Google Scholar]
  3. Gerkey, B.P.; Matarić, M.J. A formal analysis and taxonomy of task allocation in multi-robot systems. Int. J. Robot. Res. 2004, 23, 939–954. [Google Scholar] [CrossRef]
  4. Parker, L.E. Distributed intelligence: Overview of the field and its application in multi-robot systems. J. Phys. Agents 2008, 2, 5–14. [Google Scholar] [CrossRef]
  5. Matarić, M.J. Learning in behavior-based multi-robot systems: Policies, models, and other agents. Cognit. Syst. Res. 2001, 2, 81–93. [Google Scholar] [CrossRef]
  6. Wu, J.; Xu, X.; Wang, J.; He, H.G. Recent advances of reinforcement learning in multi-robot systems: A survey. Control Decis. 2011, 26, 1601–1610. [Google Scholar]
  7. Yan, Z.; Jouandeau, N.; Cherif, A.A. A survey and analysis of multi-robot coordination. Int. J. Adv. Robot. Syst. 2013, 10, 399. [Google Scholar] [CrossRef]
  8. Vig, L.; Adams, J.A. Multi-robot coalition formation. IEEE Trans. Robot. 2006, 22, 637–649. [Google Scholar] [CrossRef] [Green Version]
  9. Bhattacharjee, P.; Rakshit, P.; Goswami, I.; Konar, A.; Nagar, A.K. Multi-robot path-planning using artificial bee colony optimization algorithm. In Proceedings of the 2011 Third World Congress on Nature and Biologically Inspired Computing (NaBIC), Salamanca, Spain, 19–21 October 2011; pp. 219–224. [Google Scholar]
  10. Mohamed, N.; Al-Jaroodi, J.; Jawhar, I. A review of middleware for networked robots. Int. J. Comput. Sci. Netw. Secur. 2009, 9, 139–148. [Google Scholar]
  11. Portugal, D.; Rocha, R. A survey on multi-robot patrolling algorithms. In Technological Innovation for Sustainability; Springer: Berlin/Heidelberg, Germany, 2011; pp. 139–146. [Google Scholar]
  12. Carpin, S. Fast and accurate map merging for multi-robot systems. Auton. Robot. 2008, 25, 305–316. [Google Scholar] [CrossRef]
  13. Wang, Y.; De Silva, C.W. Multi-robot box-pushing: Single-agent Q-learning vs. team Q-learning. In Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China, 9–15 October 2006; pp. 3694–3699. [Google Scholar]
  14. Antonelli, G.; Arrichiello, F.; Chiaverini, S. Flocking for multi-robot systems via the null-space-based behavioral control. Swarm Intell. 2010, 4, 37. [Google Scholar] [CrossRef]
  15. Klavins, E. Communication complexity of multi-robot systems. In Algorithmic Foundations of Robotics V; Springer: Berlin/Heidelberg, Germany, 2004; pp. 275–292. [Google Scholar]
  16. Pugh, J.; Martinoli, A. Relative localization and communication module for small-scale multi-robot systems. In Proceedings of the 2006 IEEE International Conference on Robotics and Automation, ICRA 2006, Orlando, FL, USA, 14–19 May 2006; pp. 188–193. [Google Scholar] [Green Version]
  17. Fink, J. Communication for Teams of Networked Robots. Ph.D. Thesis, University of Pennsylvania, Philadelphia, PA, USA, 2011. [Google Scholar]
  18. Jawhar, I.; Mohamed, N.; Al-Jaroodi, J.; Agrawal, D.P.; Zhang, S. Communication and networking of UAV-based systems: Classification and associated architectures. J. Netw. Comput. Appl. 2017, 84, 93–108. [Google Scholar] [CrossRef]
  19. Murphy, R.R. Marsupial and shape-shifting robots for urban search and rescue. IEEE Intell. Syst. Appl. 2000, 15, 14–19. [Google Scholar] [CrossRef]
  20. Waharte, S.; Trigoni, N.; Julier, S. Coordinated search with a swarm of UAVs. In Proceedings of the 6th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks Workshops, SECON Workshops ’09, Rome, Italy, 22–26 June 2009; pp. 1–3. [Google Scholar]
  21. Merino, L.; Caballero, F.; Martínez-de Dios, J.R.; Ferruz, J.; Ollero, A. A cooperative perception system for multiple UAVs: Application to automatic detection of forest fires. J. Field Robot. 2006, 23, 165–184. [Google Scholar] [CrossRef]
  22. Wagner, I.A.; Bruckstein, A.M. Cooperative cleaners: A study in ant robotics. In Communications, Computation, Control, and Signal Processing; Springer: Berlin, Germany, 1997; pp. 289–308. [Google Scholar]
  23. Noguchi, N.; Will, J.; Reid, J.; Zhang, Q. Development of a master–slave robot system for farm operations. Comput. Electron. Agric. 2004, 44, 1–19. [Google Scholar] [CrossRef]
  24. Yinka-Banjo, C.; Bagula, A.; Osunmakinde, I.O. Autonomous Multi-robot Behaviours for Safety Inspection under the Constraints of Underground Mine Terrains. Ubiquitous Comput. Commun. J. 2012, 7, 1316. [Google Scholar]
  25. Stroupe, A.; Huntsberger, T.; Okon, A.; Aghazarian, H.; Robinson, M. Behavior-based multi-robot collaboration for autonomous construction tasks. In Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2005, Edmonton, AB, Canada, 2–6 August 2005; pp. 1495–1500. [Google Scholar]
  26. Maza, I.; Caballero, F.; Capitán, J.; Martínez-de Dios, J.R.; Ollero, A. Experimental results in multi-UAV coordination for disaster management and civil security applications. J. Intell. Robot. Syst. 2011, 61, 563–585. [Google Scholar] [CrossRef]
  27. Guo, Y.; Parker, L.E.; Madhavan, R. Towards collaborative robots for infrastructure security applications. In Proceedings of the 2004 International Symposium on Collaborative Technologies and Systems, San Diego, CA, USA, 18–23 January 2004; pp. 235–240. [Google Scholar]
  28. Correll, N.; Bekris, K.E.; Berenson, D.; Brock, O.; Causo, A.; Hauser, K.; Okada, K.; Rodriguez, A.; Romano, J.M.; Wurman, P.R. Analysis and observations from the first amazon picking challenge. IEEE Trans. Autom. Sci. Eng. 2016, 15, 172–188. [Google Scholar] [CrossRef]
  29. Alami, R.; Fleury, S.; Herrb, M.; Ingrand, F.; Robert, F. Multi-robot cooperation in the MARTHA project. IEEE Robot. Autom. Mag. 1998, 5, 36–47. [Google Scholar] [CrossRef] [Green Version]
  30. Kitano, H.; Asada, M.; Kuniyoshi, Y.; Noda, I.; Osawa, E. Robocup: The robot world cup initiative. In Proceedings of the First International Conference on Autonomous Agents, Marina del Rey, CA, USA, 5–8 February 1997; pp. 340–347. [Google Scholar]
  31. Khoshnevis, B.; Bekey, G. Centralized sensing and control of multiple mobile robots. Comput. Ind. Eng. 1998, 35, 503–506. [Google Scholar] [CrossRef]
  32. Ko, Y.B.; Vaidya, N.H. Location-Aided Routing (LAR) in mobile ad hoc networks. Wirel. Netw. 2000, 6, 307–321. [Google Scholar] [CrossRef]
  33. Mikki, M.A. Energy efficient location aided routing protocol for wireless MANETs. arXiv, 2009; arXiv:0909.0093. [Google Scholar]
  34. Wang, N.C.; Chen, J.S.; Huang, Y.F.; Wang, S.M.; Chen, S. A Greedy Location-Aided Routing Protocol for Mobile Ad Hoc Networks. In Proceedings of the 8th WSEAS International Conference on Applied Computer and Applied Computational Science, Hangzhou, China, 20–22 May 2009. [Google Scholar]
  35. Bhuvaneswari, P.; Vaidehi, V. Location Aided Energy Efficient Routing Protocol in Wireless Sensor Network. Int. J. Simul. Syst. Sci. Technol. 2010, 11, 41–50. [Google Scholar]
  36. Shen, H.; Zhao, L. ALERT: An anonymous location-based efficient routing protocol in MANETs. IEEE Trans. Mob. Comput. 2013, 12, 1079–1093. [Google Scholar] [CrossRef]
  37. Cordeiro, C.; Agrawal, D.P. Ad Hoc and Sensor Networks: Theory and Applications; World Scientific: Singapore, 2011. [Google Scholar]
  38. Singh, S.P.; Sharma, S. A survey on cluster based routing protocols in wireless sensor networks. Procedia Comput. Sci. 2015, 45, 687–695. [Google Scholar] [CrossRef]
  39. Hasan, M.Z.; Al-Rizzo, H.; Al-Turjman, F. A survey on multipath routing protocols for QoS assurances in real-time wireless multimedia sensor networks. IEEE Commun. Surv. Tutor. 2017, 19, 1424–1456. [Google Scholar] [CrossRef]
  40. Olenewa, J.L. Guide to Wireless Communicatinos; Cengage Learning: Boston, MA, USA, 2014. [Google Scholar]
  41. Stallings, W. Wireless Communications and Networks; Prentice Hall: Upper Saddle River, NJ, USA, 2005. [Google Scholar]
  42. IEEE 802.11, IEEE 802.16. Available online: http://en.wikipedia.org/wiki (accessed on 10 December 2014).
  43. Quigley, M.; Conley, K.; Gerkey, B.; Faust, J.; Foote, T.; Leibs, J.; Wheeler, R.; Ng, A.Y. ROS: An open-source Robot Operating System. In Proceedings of the ICRA Workshop on Open Source Software, Kobe, Japan, 12–17 May 2009; Volume 3, p. 5. [Google Scholar]
  44. Reid, R.; Cann, A.; Meiklejohn, C.; Poli, L.; Boeing, A.; Braunl, T. Cooperative multi-robot navigation, exploration, mapping and object detection with ROS. In Proceedings of the 2013 IEEE Intelligent Vehicles Symposium (IV), Gold Coast, Australia, 23–26 June 2013; pp. 1083–1088. [Google Scholar]
  45. Li, R.; Oskoei, M.A.; Hu, H. Towards ROS based multi-robot architecture for ambient assisted living. In Proceedings of the 2013 IEEE International Conference on Systems Man, and Cybernetics (SMC), Manchester, UK, 13–16 October 2013; pp. 3458–3463. [Google Scholar]
  46. Hennes, D.; Claes, D.; Meeussen, W.; Tuyls, K. Multi-robot collision avoidance with localization uncertainty. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems, Valencia, Spain, 4–8 June 2012; Volume 1, pp. 147–154. [Google Scholar]
  47. Hu, G.; Tay, W.P.; Wen, Y. Cloud robotics: Architecture, challenges and applications. IEEE Netw. 2012, 26, 21–28. [Google Scholar] [CrossRef]
  48. Turnbull, L.; Samanta, B. Cloud robotics: Formation control of a multi robot system utilizing cloud infrastructure. In Proceedings of the IEEE Southeastcon, Lexington, KY, USA, 13–16 March 2013; pp. 1–4. [Google Scholar]
  49. Kehoe, B.; Patil, S.; Abbeel, P.; Goldberg, K. A survey of research on cloud robotics and automation. IEEE Trans. Autom. Sci. Eng. 2015, 12, 398–409. [Google Scholar] [CrossRef]
  50. Wang, L.; Liu, M.; Meng, M.Q.H.; Siegwart, R. Towards real-time multi-sensor information retrieval in cloud robotic system. In Proceedings of the 2012 IEEE Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Hamburg, Germany, 13–15 September 2012; pp. 21–26. [Google Scholar]
  51. Bolodurina, I.; Parfenov, D. Development and research of models of organization distributed cloud computing based on the software-defined infrastructure. Procedia Comput. Sci. 2017, 103, 569–576. [Google Scholar] [CrossRef]
  52. Hartanto, R.; Eich, M. Reliable, Cloud-Based Communication for Multi-Robot Systems. In Proceedings of the 2014 IEEE International Conference on Technologies for Practical Robot Applications (TePRA), Woburn, MA, USA, 14–15 April 2014. [Google Scholar]
  53. Kreutz, D.; Ramos, F.M.; Verissimo, P.E.; Rothenberg, C.E.; Azodolmolky, S.; Uhlig, S. Software-defined networking: A comprehensive survey. Proc. IEEE 2015, 103, 14–76. [Google Scholar] [CrossRef]
  54. Liu, J.; Li, Y.; Chen, M.; Dong, W.; Jin, D. Software-defined internet of things for smart urban sensing. IEEE Commun. Mag. 2015, 53, 55–63. [Google Scholar] [CrossRef]
  55. Mohamed, N.; Al-Jaroodi, J.; Jawhar, I. Utilizing Fog Computing for Multi-robot Systems. In Proceedings of the 2018 Second IEEE International Conference on Robotic Computing (IRC), Laguna Hills, CA, USA, 31 January 2018–2 February 2018; pp. 102–105. [Google Scholar]
  56. Bonomi, F.; Milito, R.; Zhu, J.; Addepalli, S. Fog computing and its role in the internet of things. In Proceedings of the First Edition of the MCC Workshop on Mobile Cloud Computing, Helsinki, Finland, 17 August 2012; pp. 13–16. [Google Scholar]
  57. Mohamed, N.; Al-Jaroodi, J.; Jawhar, I. Fog-Enabled Multi-Robot Systems. In Proceedings of the 2018 IEEE 2nd International Conference on Fog and Edge Computing (ICFEC), Washington, DC, USA, 3 May 2018; pp. 1–10. [Google Scholar]
Jsan 07 00052 g001 550
Figure 1. Robot-to-Robot (R2R) and Robot-to-Infrastructure (R2I) Communication.
Figure 1. Robot-to-Robot (R2R) and Robot-to-Infrastructure (R2I) Communication.
Jsan 07 00052 g001
Jsan 07 00052 g002 550
Figure 2. Various protocols for MRS systems: (a) Robot-to-Robot (R2R) links, and (b) Robot-to- Infrastructure (R2I) Links.
Figure 2. Various protocols for MRS systems: (a) Robot-to-Robot (R2R) links, and (b) Robot-to- Infrastructure (R2I) Links.
Jsan 07 00052 g002
Jsan 07 00052 g003 550
Figure 3. The networking protocols at the various layers of the nodes using robot-to-robot (R2R) communication.
Figure 3. The networking protocols at the various layers of the nodes using robot-to-robot (R2R) communication.
Jsan 07 00052 g003
Jsan 07 00052 g004 550
Figure 4. The networking protocols at the various layers of the nodes using robot-to-infrastructure (R2I) communication.
Figure 4. The networking protocols at the various layers of the nodes using robot-to-infrastructure (R2I) communication.
Jsan 07 00052 g004
Jsan 07 00052 g005 550
Figure 5. Cloud and Fog Integrated MRS.
Figure 5. Cloud and Fog Integrated MRS.
Jsan 07 00052 g005
Table
Table 1. Wireless network protocols for MRS systems.
Table 1. Wireless network protocols for MRS systems.
ProtocolMain CharacteristicsPhysical Layer SpecsData Link Layer SpecsData RateTransmission RangeMRS System Communication and Link Types
IEEE 802.15.1 (Bluetooth)Low- to medium-rate short-range communication2.4 GHz band, FHSS/FSKMaster/Slave, Piconet/Scatternet architecture1 to 24 Mbps10 mShort-range R2R and R2I communication (e.g., task synchronization, small-size telemetric data exchange, etc.)
IEEE 802.15.3High rate cable replacement2–9 GHz and 57–66 GHz range bands, OFDM, direct sequence UWBMaster/Slave, Piconet/Scatternet architecture11–55 Mbps10 mHigh-rate short-range R2R and R2I communication (e.g., audio/video data exchange, etc.)
IEEE 802.11a/b/g/n/acLocal Area Network, medium range2.4 GHz and 5 GHz band, DSSS, OFDMCSMA/CA, DFS/PFS Mechanisms15, 30, 45, 60, 90, 120, 135, 150, 346, 800, 3466 Mbpsup to 250 m outdoorsR2R and R2I links with medium range communication and medium to high data rate.
IEEE 802.16/rel 1/rel 1.5/rel 2(m) (WiMAX)Metropolitan Area Network2 to 66 GHz band, MIMO-OFDMATDD, FDD2 to 75 MbpsUp to 35 miles (56 Km)R2R and R2I links with high data rates and longer range. Supports various types of back haul data traffic.
Cellular 3GLong range. Packet switched data. Voice support with packet or circuit switched connectivity.800 MHz to 1900 MHzCDMA, HSDPA144 Kbps (mobile users) to 42 Mbps (for stationary users)Cell radius dependent, up to several Km’sMostly R2I links to provide connectivity to infrastructure networks and the Internet in areas where other wireless services are not available.
Cellular 4G/LTEsame as 3G700 MHz to 2500 MHzLTE and LTE Advanced300 Mbps to 1 GbpsCell radius dependent, up to several Km’sMostly R2I links with higher data rate requirements to provide connectivity to infrastructure networks and the Internet in areas where other wireless services are not available.
Satellite (LEO/MEO/GEO)Wide Area Network1.53 GHz to 31 GHzFDMA and TDMA10 Mbps (upload) and 1 Gbps (download)Covers hundreds of Km’s to entire earthR2I links supporting data traffic communication to infrastructure networks and the Internet in remote areas where other connectivity is not readily available.

Share and Cite

MDPI and ACS Style

Jawhar, I.; Mohamed, N.; Wu, J.; Al-Jaroodi, J. Networking of Multi-Robot Systems: Architectures and Requirements. J. Sens. Actuator Netw. 2018, 7, 52. https://doi.org/10.3390/jsan7040052

AMA Style

Jawhar I, Mohamed N, Wu J, Al-Jaroodi J. Networking of Multi-Robot Systems: Architectures and Requirements. Journal of Sensor and Actuator Networks. 2018; 7(4):52. https://doi.org/10.3390/jsan7040052

Chicago/Turabian Style

Jawhar, Imad, Nader Mohamed, Jie Wu, and Jameela Al-Jaroodi. 2018. "Networking of Multi-Robot Systems: Architectures and Requirements" Journal of Sensor and Actuator Networks 7, no. 4: 52. https://doi.org/10.3390/jsan7040052

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop