Efficient Resource Scheduling for Multipath Retransmission over Industrial WSAN Systems

With recent adoption of Wireless Sensor-Actuator Networks (WSANs) in industrial automation, wireless control systems have emerged as a frontier of industrial networks. Hence, it has been shown that existing standards and researches concentrate on the reliability and real-time performance of WSANs. The multipath retransmission scheme with multiple channels is a key approach to guarantee the deterministic wireless communication. However, the efficiency of resource scheduling is seldom considered in applications with diverse data sampling rates. In this paper, we propose an efficient resources scheduling algorithm for multipath retransmission in WSANs. The objective of our algorithm is to improve efficiency and schedulability for the use of slot and channel resources. In detail, the proposed algorithm uses the approaches of CCA (clear channel assessment)-Embedded slot and Multiple sinks with Rate Monotonic scheme (CEM-RM) to decrease the number of collisions. We have simulated and implemented our algorithm in hardware and verified its performance in a real industrial environment. The achieved results show that the proposed algorithm significantly improves the schedulability without trading off reliability and real-time performance.


Introduction
The process automation industry continuously works towards minimizing the cost of production and maintenance while improving the quality of their products. A step towards that goal is to utilize wireless technologies, which offer advantages such as less maintenance, more flexibility, and easy deployment in harsh industrial environments [1,2]. As one of the key technologies, industrial Wireless Sensor-Actuator Networks (WSANs) will play a vital role in the Industry 4.0 framework and Cyber Physical System (CPS) and will also be important for future smart factories and intelligent manufacturing systems [3]. However, harsh conditions, such as complicated environments and metal interference, result in more packet loss in WSAN transmissions compared with traditional wireless sensor networks [4]. Additionally, application-driven industrial communication increases the requirements of stringent reliability as well as time efficiency [5]. Within process automation, standards such as WirelessHART, ISA100.11a, and IEEE (Institute of Electrical and Electronics Engineers) 802. 15.4e have been operational for nearly a decade, and there exists several researches that address support for emerging applications, such as data aggregation [6], control, and safety [7,8]. All of the above standards employ the Times-Slotted Channel Hopping (TSCH) Medium Access Control (MAC) protocol instead of pure contention-based MAC protocol Carrier-Sense Multiple Access with Collision Avoidance (CSMA/CA). The TSCH provides the capability of allocating a specific amount of bandwidth per node in a preknown pattern [9,10]. Moreover, channel hopping addresses unreliability caused by A network mainly consists of field nodes, multiple sinks, a gateway, and a centralized network manager. All field nodes operate over low-power radios compliant with the IEEE 802. 15.4-2006 standard, which supports 16 channels in the 2.4 GHz ISM band. The TSCH MAC protocol is implemented in this paper to guarantee deterministic communication. One time slot is 10 ms long and allows for channel switching and the transmission of a single packet together with data-link acknowledgement (DL-ACK). Time slots can be either dedicated or shared, where the shared time slot is amended to CCA-Embedded Slot (CES) as proposed in Reference [34]. In a dedicated time slot (DTS), only a single transmitter-receiver pair can communicate on any given channel, thereby not allowing channel reuse. CES, in which multiple nodes are scheduled on the same channel, is mainly used for decreasing the use of slots for path crossing. To guarantee the reliability of the flow transmission, we adopt the graph routing retransmission approach advised in the WirelessHART standard. As shown in Figure 2, packets can be sent by a device which has two direct neighbors at least in a graph. One is the primary routing path (PRP), and another is the alternative routing path (ARP). Each type of path has a receiving node for relaying packets. A node should preferentially send packets along the PRP. If the transmission is failed in PRP, the node will change to using ARP. In this paper, we assume that the multipath graph is already generated by the network manager. Therefore, all the possible transmissions should be involved in a slot and channel scheduling. The scheduling of slot and channel for all packet flows also has some considerations and limitations in the following based on WirelessHART.   • The allocation of slot and channel based on graph routing is so that a device can send a packet at most three times, including the first transmission and two retransmissions. The first transmission and retransmission are along the PRP, and the second retransmission needs to change to be along the ARP, where the changes of link includes the neighbor and channel. • According to the above consideration, each flow consists of dedicated and abundant transmissions, all which should be allocated for the slots and channels by the network manager. Moreover, each slot allocation should follow the routing orders, where the slot offset of next-hop transmission and retransmissions should not be allocated before the slots of the previous-hop transmission.

System Model
We describe the model of a WSAN, which consists of field devices, multiple sinks, a gateway, and a centralized network manager, as shown in Figure 1. We consider that the network can be modelled as a graph G = (N, L), in which the set N = {n 1 , n 2 , . . . , sk} represents all the nodes, including field devices n i and sinks sk. L is the link set, where the element l ij in L indicates the connection state between the nodes n i and n j . l ij = 1 means n i and n j can directly communicate with each other.
x(n i , n j ) represents a transmission with the direction from node n i to n j . Thus, for a link l ij , x t,c (l ij ) represents the transmission allocated on the time slot t and channel offset c. The length of a time slot is denoted by κ. The actual channel can be calculated by (ASN + c) mod |C|, where ASN is the absolute counting of slot advance and C is an available channel set. Thus, the channel scheduling is only relevant to the assignment of channel offset. For example, x t,c (l ij ) = 1 represents that node n i transmits a packet to node n j using channel c at slot t, otherwise, x t,c (l ij ) = 0. Based on the above network model, all transmissions need to be scheduled by the TSCH MAC protocol. Thus, some basic constraints expressed by Equations (1)-(4) should be considered, which seriously influence the performance of transmission scheduling.
First, more than one transmissions cannot be occurred simultaneously in a certain slot and channel unless nodes use some contention-based mechanism. Otherwise, one transmission may generate wireless interference to interfere with the other transmissions.
Second, the DL-ACK should be sent within one slot for reliability. Thus, a node can only transmit to one neighbor at any slot.
Third, since a low-power node in the network only supports half-duplex wireless communication, obviously, it cannot be the transmitter and the receiver simultaneously.
Finally, multichannel transmission can make full use of slot and channel resources; however, the number of available channels is limited. Thus, the number of transmissions cannot be beyond the number of available channels.
We consider all the end-to-end deliveries in a network as a flow set, which is denoted by F = {F 0 , F 1 , . . . , F γ }. The number of flows that need to be scheduled is denoted as γ. A flow can be characterized by F k : . . } is the collection of all transmissions in a flow, E k is the collection of transmission release orders related to the routing order, and φ k is the routing path from the first source node to the last destination node. s τ and d τ are the source node and the destination node in a transmission, respectively. One of the elements e(α, β) in E k represents that the transmission x k (s β , d β ) is released after all the transmissions x k (s α , d α ). Each F k periodically generates a packet with the period p k , and the flow should be finished before its deadline d k .
All the slots for a flow construct a superframe which is repeated periodically. The supported periods are defined as 2 a · p m based on WirelessHART, where a is an integer and p m is the minimum period among all flows. Without loss of generality, we transfer the period of all flows to the regular period through Equation (5), since it can let the scheduling fit with the data period better.
For example, assuming the basic period is 20 ms, then the supported periods can be 20 ms, 40 ms, 80 ms, and so on. All the superframes constitute a hyperframe, the period of which is the maximum period among the superframes. The period is denoted as T and is expressed by Equation (6).
We use H[t][c] to denote which transmission is allocated on time slot t and channel c. The number of slots in a hyperframe is denoted by |H| = T/κ. As an example, if the first transmission of flow F 1 is allocated for slot 0 and channel offset 0, it can be expressed as . We assume that the deadline of a flow transmission is equal to its period in this paper. Thus, all the transmissions in a flow should be scheduled within the period, where the constraint can be expressed by Equation (7).

The Proposed Scheduling
In this section, we introduce the CEM-RM algorithm for the multipath retransmission scheme based on graph routing in WirelessHART. We also present the proposed approaches to make the scheduling more efficient. The symbols and related functions used in the algorithms are listed in Table 1.

Symbols Description
D i the number of links l ji in a subgraph, where the node n i is the destination node.

H[t][c]
the transmission allocated for time slot t and channel c W the collection of time slots, which are already allocated for transmission τ, where its element is τ C t the collection of available channels at time slot t Ω t the collection of all the transmissions allocated to time slot t Ψ t the collection of available sinks at time slot t |C t |, |Ψ t | the number of corresponding elements in the collection

Functions Description
Add(y, Y) the function responsible for adding an element y into the collection Y Del(y, Y) the function responsible for deleting an element y into the collection Y ASort(Y, z) the function responsible for sorting all the elements in Y in ascending order, according to feature z

Transmissions Release Algorithm
As mentioned before, the multipath scheme requires that a packet on a device is transmitted three times at most, including initial transmission and two retransmissions. Thus, we first need to generate all the transmissions and their corresponding orders for a flow before allocating slots and channels. Algorithm 1 is the function of TransmissionRelease(F), which can generate all transmissions and transmission orders for a flow, according to a given graph G. Actually, the period p and the routing φ of the flow are already known and we mainly focus on producing the collections X and E to complete the flow. The initial transmission and the first retransmission use PRP, and the second retransmission uses ARP. A link in φ releases two transmissions if it is used for PRP; otherwise, it releases only one transmission for ARP. To obtain all the transmissions, we traverse all the links in φ from the source node to the destination node by a first in and first out (FIFO) pipe. Meanwhile, to generate the transmission order, we give each n i a transmission collection X (n i ), which involves the already generated transmissions. The destination node of these transmissions is n i . A node will be put into the FIFO and obliged to release the transmission until the number of transmissions in its collection X (n i ) is equal to D i . D i is the in-degree of n i , which can be calculate by Equation (8).
To illustrate the function of TransmissionRelease(F), we consider the topology shown in Figure 2 as an example. The flow is generated by node n 1 and transmitted to node n 10 . As shown in Figure 3, the generated transmissions in X are in released sequence after executing Algorithm 1. For example, the first released transmission is x(n 1 , n 2 ), and the fourth is x(n 2 , n 6 ). The transmissions in X also need sequence collection E to further explain their routing order, where e(2, 4) represents that the 4th transmission x(n 2 , n 6 ) must occur after the second transmission x(n 1 , n 2 ). The transmissions without orders can occur simultaneously, which means they can be allocated for the same slot with different channels. We can see that the two adjacent transmissions may not be in continuous order, such as the 6th transmission x(n 2 , n 4 ) and 7th transmission x(n 3 , n 4 ). We also notice that E has two orders, e(5, 18) and e (12,18), where the transmission x(n 6 , n 10 ) should go after the 5th and 12th transmissions. Actually, we only need to comply with the latter order, e (12,18), which should be considered in slot and channel allocation.

CEM-RM Algorithm
The proposed scheduling algorithm CEM-RM is shown in Algorithm 2. Considering decreasing the negative effect of the diverse period, path crossing, and conflict, the algorithm adopts three approaches to improve schedulability.
Assuming that t α and c α respectively represent the slot offset and channel offset allocated to the transmission x k (s α , d α ) within a hyperframe. For a flow F k , if there is e(α, β), the slot allocation must satisfy t α < t β . Meanwhile, all the slots t α + j · p k (j ≥ 0) are allocated to the transmission x k (s α , d α ) due to periodic characteristics. Since there exists node interference in the time slot, the objective of the scheduling is to avoid these interference. However, the slot allocation for transmissions has different periods. This causes more node interference, which reduces channel utilization. The given network graph is schedulable (SCH) when the slot allocation of all the flows satisfies the above constraints expressed by Equations (1)-(4) and (7); otherwise, it is unschedulable (UNSCH). As a classical scheduling strategy, rate monotonic (RM) policy preferentially assigns communication resources to the flow which possesses the shorter period [35]. We adopt RM policy as a basic strategy, where we sort all the flows in ascending period at the beginning (line 2).
After individually tracing the RM-based scheduling, we notice that more transmissions need to be scheduled for a node that is close to the gateway, especially for the sink node. As shown in Figure 4, obviously, the sum of available communication resources is |H| · |C|, where we regard a slot with a channel as a communication resource. On the one hand, the generated transmissions of all flows cannot exceed the number of communication resources, which is expressed as follows: On the other hand, the number of available communication resources is p k · |C| when the flow F k ∈ F is to be scheduled. However, some of these available resources are occupied by the previous k − 1 allocated flows. The condition that there are enough available resources for F k is expressd as follows: q represents all the integers within [0, T/p k ); Del(c, C t+q·p k ); 28 τ ← t; τ ← t + 1; The available resources are becoming more inadequate with the increase of data flow. The later scheduled flows will meet more node interferences, especially at the end hop of a flow. This is because most transmissions share the sink node. A sink node cannot be involved in two transmissions simultaneously at a slot, which results in low schedulability. Moreover, the interference caused by one sink node makes it so that only one channel can be scheduled at the end of a hyperframe. Thus, we adopt multiple sinks to reduce the interference caused by a single sink node during hyperframe scheduling. Each slot possesses multiple available sinks to receive packets on different channels. In CEM-RM, a transmission may suffer interference at one slot, which is caused by a sink node. The algorithm will use the other available sink to avoid such interferences if at least one resource exists at the slot (line 19).
Furthermore, one key adjustment for the slot employed in this paper is that we amended the common shared slot to the CES. The multipath scheme provides an alternative relay node for possible retransmissions. Actually, a packet is finally routed along one of these paths. Redundant resources need to be assigned for avoiding conflict when the different routing paths share the same relay node. Thus, merging the resources for the overlapping paths is an effective approach. However, the conflict may be caused by the DL-ACK loss problem. A packet is simultaneously transmitted along more than one path if one of the transmissions only loses a DL-ACK. The packet is still transmitted along the original path. Considering this phenomenon deteriorates the reliability and costs extra energy, we adopt an efficient approach of the CES.
At a CES, the timing structure of the transmitting node is shown in Figure 5 and the destination node still accords with the standard. The main adaptation is to add CCA units (128 µs) in the TsTxOffset, which is the 2120 ± 100 µs duration between the beginning of the slot and the start of the data frame transmission. We respectively reserve 200 µs and 340 µs at the beginning and end of TsTxOffset for some preparation of the slot and data frame. In CCA units, we adopt CCA mode 2 in IEEE 802.15.4 standard, where CCA shall report a busy medium upon the detection of a signal with the modulation and spreading characteristics of an IEEE 802.15.4 compliant signal. The maximum number of CCA units is 5 in this paper. Multiple source nodes will contend this slot by randomly selecting a CCA unit to detect the state of a channel. Once a source node detects that the channel is free, it sends a preamble signal to occupy the current slot during the remaining time TsTxOffset. Another source node will detect the preamble; then, it stops to perform CCA and to give up using the current slot. The radio chip may switch the state and channel in TsRxTx, which is less than 192 µs. We assume that there are K transmissions with different source nodes and the same destination node. The probability of successful contention is (K/5) · ∑ 4 j=0 (j/5) K−1 , K ≥ 0 and enough to guarantee the reliability of the flow transmission, since the contention at CES occurs only when the DL-ACK is lost on the previous link. The use of CES is to decrease abundant slot allocation and to increase slot use, which also results in the increase of schedulable ratios. With the use of the above three approaches, we designed the slot and channel allocation to avoid node interference. The following two situations should be solved by the above approaches when the allocation to a transmission x k 1 (s α , d α ) meets the already allocated transmission x k 2 (s β , d β ) in the same slot.
(1) According to constraints expressed by Equations (1)-(3), two different transmissions (α = β) in a flow (k 1 = k 2 ) should be allocated for different slots if the source node of a transmission is the same as one of the nodes in another transmission. However, if d α = d β and two transmissions are allocated for the same slot exactly, those transmissions can use the same channel and the slot is changed into a CES. With the assumption of α > β, the condition can be expressed as follows: (2) Accordingly, two transmissions in different flows (k 1 = k 2 ) also should be allocated for different slots if the source node of a transmission is the same as one of the nodes in another transmission. Also, if d α = d β = sk, CES cannot be used for the transmissions in different flows. However, if d α = d β = sk, we can change d α to one of the other available sink nodes. In this scenario, the above two transmissions can be allocated at the same slot with different channels. With the assumption of p k 1 > p k 2 , q represents all integers within [0, p k 1 /p k 2 ). Therefore, the condition can be expressed as follows: if The above two conditions represent the situations of node interference and scheduling interference in slot and channel scheduling. The difference between the designed algorithm and RM algorithms is that we assign the communication resources by the order of each flow, instead of the slot sequence. Thus, we traverse all the transmissions in each period-sorted flow (lines 3 and 6) and assign the slots and channels to each transmission. To avoid interference, the scheduling should be adjusted when the allocation meets these conditions. We add all assigned transmissions at time slot t into Ω t . We define C t and Ψ t as the collections of available channels and sinks at time slot t, respectively. Their elements will be removed by the Del() function when the corresponding channels or sinks have already assigned. Moreover, τ is used to record the slot offset allocated to the transmission τ in every flow, and it is responsible for finding the earliest available slot which does not break the routing order (lines [11][12][13]. There could be several transmissions at a CES. Thus, changing a normal slot to a CES means that the slot expands for multiple transmissions. With the above considerations, we can finally obtain the scheduling table H when all transmissions are scheduled within their period. In the algorithm, we define |N F | and |X F | as the maximum number of nodes and transmissions, respectively, and they are all related to the length of the flow. For example, for a four-hop flow in a routing tree topology, |N F | is less than 24. All the nodes have a PRP except the sink node and have an ARP except the first-hop nodes and the sink node. Thus, |X F | is less than 2 × (2 4 − 1) + 1 × (2 3 − 1) = 37.  p m ), respectively. Therefore, the time complexity in a worst case is O(γlog(γ) + γ(|N F | 3 + |H||X F | + |H| · T/p m )). In practicality, the periods of different applications could not have much difference; thus, |H| · T/p m could be ignored.

Simulations and Experiments
In this section, we evaluate the performance of the proposed algorithm by comparing with two widely used real-time scheduling policies, RM and Least Laxity First (LLF) [36]. RM schedules a transmission based on the packet's absolute deadline, while LLF schedules a transmission based on the packet's laxity, which is defined as the transmission's deadline minus remaining transmission time. The schedules of individual RM and LLF are not suitable with our applications due to the quantity of nodes and period. Therefore, we also add multiple sinks into each compared scheduling policy for fairness. Thus, we respectively denote the two schedules as M-RM and M-LLF in the following simulations and experiments.
Different topology structures need different quantities of communication resources. Thus, four kinds of topology structures are generated for comparison, which are denoted as Tp1, Tp2, Tp3, and Tp4. We assume that the furthest leaf node is four hops away from the gateway in all topology structures. In industrial wireless networks, since there is the requirement of reliable and real-time performance, on the one hand, the big hop size can increase the rate of packet loss and, on the other hand, the large network cannot cater the demand of transmission deadlines. In our experience, four hops can overlap the general factory. Meanwhile, four hops can exactly overlap the experimental factory in this paper. The difference between each topology structure is that the number of nodes on each hop is generated according to different probabilities. For example, to generate one of the Tp1 topologies, we first traverse all the nodes, where the total number of nodes is determined previously. The traverse approach stipulates that a node has a 0.3 probability of being on the first hop, a 0.3 probability of being on the second, a 0.3 probability of being on the third, and a 0.1 probability of being on the fourth. Then, each node on the lower hop randomly selects two higher-level nodes as their parents to be the major path and redundant path. Thus, all the topologies of the Tp1 structure may be different from each other. Without loss of generality, we will stochastically generate enough topologies of every kind of structure. With the same method, the other kinds of topologies are generated according to the corresponding probabilities, which are listed in Table 2.

Simulations
Our simulations are presented to demonstrate the schedulability and effectiveness of our scheduling. The algorithms have been written in C language, and the simulations have been performed on a Windows machine with a 3.1-GHz Inter Core 3 processor. The following metrics are used for performance analysis: (a) Schedulable ratio is measured as the percentage of test cases for which a scheduling policy is able to schedule all transmissions. (b) Scheduling time is the total time required to successfully make a complete hyperframe for all transmissions. (c) Average normalized bandwidth is the ratio between the number of allocated resources and the total resources within a hyperframe after scheduling. In simulations, we assume that every node generates a packet periodically and sends it to the gateway along the routing graph. A packet is regarded as a flow without aggregation. The period is randomly selected in the set {p m · 2 a | ∀a ∈ [0, b]}, where b is the maximum exponent. A packet belonging to a fourth-hop node needs at most 22 transmissions using the scheduling of multipath scheme; thus, three values of p m (0.25 s, 0.5 s, 1 s) are tested in the simulations. Figure 6 shows the schedulable ratio of the three scheduling policies with different parameter configurations. The number of nodes ranges from 50 to 100. We randomly generate 8000 cases for each node quantity and use the three algorithms to schedule the generated topologies separately. The number of available channels is set to be the maximum, 16, for a better comparison, since several cases using less channels are hardly scheduled with the existence of diverse periods. In Figure 6a-d, the three algorithms, which execute on each type of topology with p m = 1 s and b = 0, can successfully schedule the majority of graphs. Obviously, the schedulable ratio drops in pace with the increase of nodes; however, we can see that our proposed scheduling is almost unaffected with these configurations. With the period shortened and diversified, the number of schedulable graphs decreases dramatically. The execution time of a scheduling can affect the performance of network formation and rescheduling; thus, we have measured the average run time of the three algorithms. Each test with a different number of nodes and algorithms includes 100 successfully scheduled graphs. To guarantee that the tested graphs are more schedulable, we set p m = 0.5 s and b = 1 in all tests. As displayed in Figure 7, the markers represent the average time of different algorithms with different numbers of nodes and the lines show the trend of scheduling time as the number of nodes is increased. We can obtain that the average execution times of M-LLF and M-RM are lower than 1264 ms, while the time of our proposed scheduling is lower than 371 ms. The results indicate that CEM-RM is better for the performance improvement of network formation or rescheduling.  Moreover, Figure 8 shows the average requirement of the wireless bandwidth for our scheduling and for the others. The normalized bandwidth is calculated by the ratio between the number of allocated resources and total resources in a hyperframe. We also select the successfully scheduled hyperframe with 100 nodes and the configuration of p m = 0.5 s, b = 1. As can be seen, the average normalized bandwidths of Tp4 of M-LLF and M-RM are 0.918 and 0.906, respectively, which are similar to each other, while CEM-RM only requires 0.627. The same conclusion also applies to other types of topologies. This is because the number of transmissions is determined by a graph in M-LLR and M-RM. However, CEM-RM adopts CES to solve the problem of overlapping paths and to reduce the phenomenon of DL-ACK loss, which can decrease the number of dedicated slots additionally. The remaining unscheduled resources can be allocated for network management to improve network stability.

Experiments
We also test the network performance of the scheduling policies in practice, where the performance includes the reliability and end-to-end packet delay. Since the scheduling of M-LLF and M-RM follows the basic policy (the number of retransmissions) in WirelessHART, their reliability and end-to-end packet delay are only affected by the channel condition. Thus, we make comparisons only between our scheduling and M-RM. The algorithms are executed on a developed WSAN system, which is compatible with WirelessHART and also supports our applications. As shown in Figure 9, the experiments are conducted in a 50 × 30 m 2 factory in which the packet error rate is about 8% mainly caused by metal and electromagnetic interference. Nodes Gateway Figure 9. Experimental environment and deployment of field devices.
Our wireless nodes used to collect corresponding information are implemented on a LPC1769 mote with an AT86rf231 radio transceiver. In reality, we have three kinds of information to be collected, which are temperature and humidity (HT) with a 1-s period, electrical information (EI) with a 0.5-s period, and machine data (MD) with a 0.25-s period. Each kind of information is collected by different versions of wireless node, respectively shown in Figure 10a-c. A total of 60 nodes consist of 30 HT nodes (HTN), 15 EI nodes (EIN), and 15 MD nodes (MDN). The information of MD is collected through a CAN (Controller Area Network) bus, and the EI is transmitted by RS485. The radio transmission power is 3 dBm, and we use 10 of the available channels with lower packet error rates. The multi-sink-based gateway with 8 sinks, a hard disk, and a wireless adaptor is shown in Figure 10d. The communication between multiple sinks and the gateway is constructed by USB port. The metal case is used to avoid dust and water to increase the lifetime. The generated topology is shown in Figure 11, and the remaining experimental parameters are listed in Table 3.  On the one hand, we first measure the packet delivery ratio (PDR) of M-RM and CEM-RM, as shown in Figure 12. The PDR represents the ratio between the number of packets that succeed in arriving to the gateway on time and the total number of generated packets. Since the period of packet generation is known previously, we can get the total number of packets of each node. Then, we gather the statistic of the number of packets that successfully reach the gateway within their own deadline. An experiment remains 20 min, and we assess the PDR every minute. The average PDRs of M-RM and CEM-RM for 20 min are 0.971 and 0.973, respectively. We can observe that their PDR for every minute vibrates over their average values because the condition of channels may change with time. The jitter of our CEM-RM is a little higher than that of M-RM, where the minute-PDR standard variations of M-RM and CEM-RM are 0.0042 and 0.0055, respectively. The time-varying channel has an effect on the adopted CES to a certain extent. However, the average PDR is mainly the same, which means that the schedulability of our proposed scheduling is improved but does not sacrifice reliability. On the other hand, we measured the delay of end-to-end delivery between each hop node and the gateway. The starting time of a packet is recorded at the time of its first transmission, while the ending time is recorded at the packet reception in the gateway. Figure 13 shows the comparison of average end-to-end delays between M-RM and CEM-RM. Obviously, the delay of CEM-RM is generally lower than that of M-RM. Moreover, as the routing hop increased, the average delay of our proposed scheduling was much lower than that of M-RM. For the packets of four-hop nodes, the average delay of our scheduling is about 32 ms lower than that of M-RM. The use of CES and multiple sinks effectively decreases the end-to-end delay, and CES can also reduce the impact of the DL-ACK loss phenomenon. The experimental results roughly indicate that our proposed scheduling not only improves the efficiency of scheduling but also decreases the end-to-end delay.

Conclusions
In this paper, we study the problem of how to improve the schedulability of communication resources in industrial WSANs. Existing centralized approaches for multipath retransmission scheme do not consider schedulability in the coexistence of diverse industrial applications. In this paper, we propose a resources-scheduling algorithm based on the multipath retransmission scheme in the WirelessHART standard. We use the approaches of RM policy, CES, and multiple sinks to reduce the negative impact of diverse periods and to avoid node interference as far as possible. By simulation and implementation in hardware, we compare our proposed CEM-RM algorithm with M-LLF and M-RM scheduling policies. The achieved results show that our proposed solution outperforms the aforementioned scheduling policies in terms of less use of communication resources, end-to-end delay, and increase in schedulable ratio without jeopardizing network reliability and timeliness of data packets.

Conflicts of Interest:
The authors declare no conflict of interest.