A Period-Aware Routing Method for IEEE 802.1Qbv TSN Networks

: The IEEE 802.1Qbv standard provides deterministic delay and low jitter guarantee for time-critical communication using a precomputed cyclic transmission schedule. Computing such transmission schedule requires routing the ﬂows ﬁrst, which signiﬁcantly affects the quality of the schedule. So far off-the-shelf algorithms like load-balanced routing, which minimize the maximum scheduled trafﬁc load ( MSTL ), have been used to accommodate more time-triggered trafﬁc. However, they do not consider that the bandwidth utilization of periodic ﬂows is decentralized and their criteria for bottleneck of scheduling are imprecise. In this paper, we ﬁrstly explore the combinability among different periods of ﬂows, which can measure their ability to share bandwidth without conﬂict. Then, we propose a novel period-aware routing algorithm to reduce the scheduling bottleneck, thus more ﬂows can be accommodated. The experiment results show that the success rate of scheduling is signiﬁcantly improved compared to shortest path routing and load balanced routing.


Introduction
In the field of industrial control and automotive networks, critical traffic such as realtime control information requires bound low delay and low jitter. However, the traditional Ethernet lacks the guarantee of real-time and certainty of communication. Although the rising real-time Ethernet can provide certain guarantees in real-time and certainty, there are some defects such as inconsistent standards and unclear development direction [1][2][3][4]. To address this limitation, the IEEE time sensitive network (TSN) task group has developed a set of unified standards based on AVB (audio video bridging) protocol, which is not only compatible with traditional Ethernet, but also ensures real-time performance and low jitter. These standards consist of time synchronization, paths selection, path reservation, fault tolerance, scheduling, traffic shaping and so on [5,6]. As part of the TSN standards, it is committed to providing deterministic network delay and low jitter guarantee for time triggered (periodic) traffic flow in real-time control, and can be compatible with best-effort flows and AVB flows [7].
To realize the time-aware scheduling defined in 802.1Qbv, precise time synchronization protocols, such as IEEE 802.1AS, are required. Such protocols enable all bridges and end stations in the network to synchronize their local clocks with an accuracy of less than 1 µs. A pre-computed communication schedule is sent to every end station in the system, instructing them when to send which frame to the network, and the end station executes the communication schedule precisely [8][9][10]. In this way, communication interference

•
We first analyze the combinability among various periods of flows and show the ability of different periods of flows to utilize the decentralized bandwidth.

•
We introduce a new metric to measure the bottleneck of scheduling. Moreover, we propose a period-aware routing algorithm for time-triggered flows and improve their schedulability. • Finally, we evaluate the proposed algorithm from three dimensions: Load, topology and flow type. The experiment results demonstrate that our method can achieve higher schedulability compared to the shortest path algorithm and the load balanced algorithm.
The rest of the paper is organized as follows. The related work is discussed in Section 2. In Section 3, we describe the system model and the problem under consideration. Then the motivation case is presented in Section 4. In Section 5, we analyze the combinability among flows in detail, based on which we proposed a period-aware routing algorithm for time-triggered flows. We evaluate our algorithm using an ILP-based schedule platform in Section 6. Finally, we conclude in Section 7.

Related Work
Routing algorithms have been well studied and some advanced results have been proposed [17,18]. Nevertheless, the contributions to TSN routing schemes only gradually emerged ten years ago [3]. Rapid spanning tree protocol and shortest path algorithm are widely used for TSN, adopting the convention of typical Ethernet [19]. Furthermore, the IEEE 802.1Qca provides explicit control over routing and mechanisms and recommends constrained shortest path first routing for TSN transmissions [20,21]. However, as previously described, shortest path routing is likely to reduce the schedulability for time-triggered flows.
In recent studies, some works [22,23] extended the routing problem to the topology planning problem. While in practical applications, the greater demand is to find the best routing scheme based on the given topology. In addition, some researches [8,24] focused on jointly solving the routing problem and scheduling problem. Although such a method can achieve better scheduling quality, it is too time-consuming for large-scale networks. Consequently, most of the methods are to solve the routing problem first, and then solve the scheduling problem.
There are many optimization goals for routing problems, such as scalability, slack, delay, et cetera [25][26][27], but the core issue is schedulability, because if schedulability cannot be guaranteed, other indicators become meaningless. In terms of schedulability, the main consideration is how to distribute the load as evenly as possible. This is the so-called load balance. Arif and Atia [28] proposed a load balanced routing model for TSN, but it focused on improving the circuit delay rather than schedulability. Nayak et al. [5] proposed two ILP-based routing algorithms to optimize the MSTL. However, the runtimes of the ILP-based method are too long, and thus detrimental to scalability. Load balanced routing heuristics are developed in literature [3,4], but they did not consider different periods of flows. Therefore, it does not meet the needs of practical application where different control devices transmit the messages at different frequencies. Although "period" has been considered as a factor for path selection in literatures [6,8], they did not recognize the decentralized bandwidth utilization of periodic flows and they just changed the metric to evaluate MSTL by taking "period" into account. Therefore, these methods are not accurate in measuring the scheduling bottleneck. In contrast, we find that flows with different periods have different abilities to utilize the dispersed bandwidth, and use the concept "combinability" to evaluate it. By carefully analyzing the combinability among flows with different periods, we propose a heuristic to reduce the scheduling bottleneck and increase the probability of a set of time-triggered flows to be successfully scheduled.

System Model and Problem Definition
In this section, we present our system model and the problem that we focus on. The main notations that we use are listed in Table 1.

Notation Explanation src
The source node of a flow. dst The destination node of a flow.

siz
The size of a flow measured by the length of time consumed during propagation. prd The cycle period of a flow.

MSTL
The maximum scheduled traffic load.

LCM
The least common multiple of the periods of all flows.

GCD
The greatest common divisor of the periods of all flows.

BW u
The used bandwidth.

BW t
The total bandwidth, BW r The residual bandwidth. num The number of flows that can be accommodated. wgt The specific weight of a flow. SOW Sum of wgt.

MSOW
The maximum sum of wgt. hops The number of edge/link in a path.

System Model
We model the time-sensitive network as a directed graph G = (V, E), where the set V = ES ∪ S consists of a finite set ES of end stations and S of switches or bridges, and the set E = {(vi, vj) | vi, vj ∈ V, and vi, vj are connected.} includes a set of full duplex edges or links. Note that each full duplex edge is regarded as two separate directional edges (vi, vj) and (vj, vi), where the former node is the source and the latter node is the destination. We assume the whole network is precisely synchronized. An example of network topology is shown in Figure 1.

System Model
We model the time-sensitive network as a directed graph G = (V, E), where the set V = ES ∪ S consists of a finite set ES of end stations and S of switches or bridges, and the set E = {(vi, vj) | vi, vj ∈ V, and vi, vj are connected.} includes a set of full duplex edges or links. Note that each full duplex edge is regarded as two separate directional edges (vi, vj) and (vj, vi), where the former node is the source and the latter node is the destination. We assume the whole network is precisely synchronized. An example of network topology is shown in Figure 1. A set of unicast time-triggered flows F is considered and the flow mentioned later refers to time-triggered flows in default. These flows are transmitted within their own cycle and the hyper cycle (the least common multiple of the periods of all flows). When all flows are released simultaneously, we can safely restrict our attention to a single hyper cycle (the first one) [3]. Each flow ∈ F is denoted by a 4-tuple f ≡ (src, dst, siz, prd), where src, dst ∈ V are the flow's source and destination node, respectively. Siz defines the size of the flow by the length of time consumed during propagation. For example, the bandwidth of TSN is 100 Mb/s and the length of the flow is 100 Byte, then the "siz" for the flow is 8 μs. Moreover, prd is the cycle period of the flow. For simplicity, we assume the minimum time unit of our system is 1 μs and the unit for siz and prd is microsecond by default. Since we mainly focus on the schedulability rather than latency, we assume that any valid solution meets the flow's deadline.

Problem Definition
The considered flows must be transmitted based on a commonly used no-wait schedule [5,6,15]. Given the network topology and a set of flows with different periods, the routing algorithm finds a path for every given flow from its source to destination, aiming to improve the success rate of scheduling.

Motivation Example
In this section, we analyze the shortcomings of the existing load balanced routing algorithm [1,3,4] to better illustrate our motivation.
When selecting the path for a flow, we mainly focus on how to reduce the bandwidth utilization of the bottleneck edge. Maximum scheduled traffic load (MSTL) is regarded as the bottleneck in existing methods [3][4][5] and they proposed different algorithms to minimize the MSTL, which is computed as A set of unicast time-triggered flows F is considered and the flow mentioned later refers to time-triggered flows in default. These flows are transmitted within their own cycle and the hyper cycle (the least common multiple of the periods of all flows). When all flows are released simultaneously, we can safely restrict our attention to a single hyper cycle (the first one) [3]. Each flow ∈ F is denoted by a 4-tuple f ≡ (src, dst, siz, prd), where src, dst ∈ V are the flow's source and destination node, respectively. Siz defines the size of the flow by the length of time consumed during propagation. For example, the bandwidth of TSN is 100 Mb/s and the length of the flow is 100 Byte, then the "siz" for the flow is 8 µs. Moreover, prd is the cycle period of the flow. For simplicity, we assume the minimum time unit of our system is 1 µs and the unit for siz and prd is microsecond by default. Since we mainly focus on the schedulability rather than latency, we assume that any valid solution meets the flow's deadline.

Problem Definition
The considered flows must be transmitted based on a commonly used no-wait schedule [5,6,15]. Given the network topology and a set of flows with different periods, the routing algorithm finds a path for every given flow from its source to destination, aiming to improve the success rate of scheduling.

Motivation Example
In this section, we analyze the shortcomings of the existing load balanced routing algorithm [1,3,4] to better illustrate our motivation.
When selecting the path for a flow, we mainly focus on how to reduce the bandwidth utilization of the bottleneck edge. Maximum scheduled traffic load (MSTL) is regarded as the bottleneck in existing methods [3][4][5] and they proposed different algorithms to minimize the MSTL, which is computed as Since we only focus on how the flows are transmitted within a hyper cycle, the hyper cycle can be regarded as the total bandwidth and it will be allocated to the flows. Actually, Equation (2) computes the maximum bandwidth used in a hyper cycle. Letting BW u be the used bandwidth, we have According to existing methods, the smaller the MSTL is, the more bandwidth is left, and the more flows it can hold, because they assume that the remaining bandwidth can be fully utilized. As is shown in Figure 2a, it is reasonable when the cycle is not considered and the flows can be tightly bunched. Nevertheless, when we take the cycle into consideration, it is inappropriate to use the residual bandwidth to measure the capacity to accommodate flows, because they are decentralized as is shown in Figure 2b. Therefore, the remaining bandwidth may be useless for some flows.
Since we only focus on how the flows are transmitted within a hyper cycle, the hyper cycle can be regarded as the total bandwidth and it will be allocated to the flows. Actually, Equation (2) computes the maximum bandwidth used in a hyper cycle. Letting BWu be the used bandwidth, we have According to existing methods, the smaller the MSTL is, the more bandwidth is left, and the more flows it can hold, because they assume that the remaining bandwidth can be fully utilized. As is shown in Figure 2a, it is reasonable when the cycle is not considered and the flows can be tightly bunched. Nevertheless, when we take the cycle into consideration, it is inappropriate to use the residual bandwidth to measure the capacity to accommodate flows, because they are decentralized as is shown in Figure 2b. Therefore, the remaining bandwidth may be useless for some flows.  Let BW t be the total bandwidth, the theoretical residual bandwidth (denoted as BW r ) can be computed as For aperiodic flows, it can be calculated that BW r = 8 in Figure 2a, which means that it can accommodate a flow with siz up to 8 or several flows with a total siz up to 8. While for periodic flows shown in Figure 2b, it can be calculated that BW r = 7 according to Equation (4). Three potential flows are added: (1) F5: siz = 1, prd = 6; (2) F6: siz = 1, prd = 3; (3) F7: siz = 3, prd = 18, whose BW u are 3, 6, 3 respectively. It is obvious that they are all less than BW r , but only F5 can be accommodated.
This shows that more bandwidth remaining does not mean that more flows can be accommodated. The capacity of different flows to utilize specific remaining bandwidth is different. In other words, some flows are not prone to conflict when sharing the same edge, while others are prone to conflict. Existing methods only focus on bandwidth distribution Electronics 2021, 10, 58 6 of 19 and ignore this characteristic of different flows. Therefore, we need a new criterion to show whether a specific flow can be accommodated on an edge.

Proposed Solution
In this section, we propose a period-aware routing heuristic for time-triggered flows. "Period-aware" means that the period of the flow is regarded as the essential element of path selection. First of all, we proposed the concept of combinability, which helps us to understand the ability of different periods of flows to share bandwidth without conflict. After that we analyze the combinability of different periods of flows and put forward a new metric to measure the bottleneck of scheduling. Then we give an example to illustrate the advantages of our method compared to the load balanced algorithm proposed in literature [3]. Finally, the procedure of our algorithm is described in detail.

The Combinability of Different Periods of Flows
When more than one time-triggered flow is passing through the same edge, the scheduling algorithm must assure that the transmission time for these flows is not overlapped to avoid congestion on that edge [2]. We put forward the concept of "combinability" to measure the capacity of flows to share bandwidth without conflict. More specifically, high combinability means that flows transmitted via the same edge are not likely to congest, bandwidth can be used more fully and this edge is able to accommodate more flows. It should be pointed out that when flows pass through the same edge, the following two restrictions need to be met:

•
The transmission times of flows are not overlapped in one hyper cycle [6]. • The first frame of each flow should be sent within its own period [5].
This shows that more bandwidth remaining does not mean that more flows can be accommodated. The capacity of different flows to utilize specific remaining bandwidth is different. In other words, some flows are not prone to conflict when sharing the same edge, while others are prone to conflict. Existing methods only focus on bandwidth distribution and ignore this characteristic of different flows. Therefore, we need a new criterion to show whether a specific flow can be accommodated on an edge.

Proposed Solution
In this section, we propose a period-aware routing heuristic for time-triggered flows. "Period-aware" means that the period of the flow is regarded as the essential element of path selection. First of all, we proposed the concept of combinability, which helps us to understand the ability of different periods of flows to share bandwidth without conflict. After that we analyze the combinability of different periods of flows and put forward a new metric to measure the bottleneck of scheduling. Then we give an example to illustrate the advantages of our method compared to the load balanced algorithm proposed in literature [3]. Finally, the procedure of our algorithm is described in detail.

The Combinability of Different Periods of Flows
When more than one time-triggered flow is passing through the same edge, the scheduling algorithm must assure that the transmission time for these flows is not overlapped to avoid congestion on that edge [2]. We put forward the concept of "combinability" to measure the capacity of flows to share bandwidth without conflict. More specifically, high combinability means that flows transmitted via the same edge are not likely to congest, bandwidth can be used more fully and this edge is able to accommodate more flows. It should be pointed out that when flows pass through the same edge, the following two restrictions need to be met: • The transmission times of flows are not overlapped in one hyper cycle [6]. • The first frame of each flow should be sent within its own period [5].

Examples for Different Combinability
For example, there are two flows passing through the same edge: (1) F1: siz = 1, prd = 3; (2) F2: siz = 1, prd = 6. The hyper cycle of these two flows is 6, which means that we only need to care about time 0 to time 6. Supposing F1 is sent at time 0, the possible situations of how F2 is transmitted under the above restrictions is shown in Figure 3.    The figure shows that there are up to four feasible sending methods for F2. Moreover, it is not difficult to find out that the edge can accommodate up to four F2 and bandwidth can be fully used. Therefore, there is a high combinability between F1 and F2.
The figure shows that there are up to four feasible sending methods for F2. Moreover, it is not difficult to find out that the edge can accommodate up to four F2 and bandwidth can be fully used. Therefore, there is a high combinability between F1 and F2.
Instead, we consider these two flows: (1) F3: siz = 1, prd = 3; (2) F4: siz = 1, prd = 6. The hyper cycle is 12 and the possible situations for F4 is shown in Figure 4 assuming F3 is sent at time 0.  We can see that F3 and F4 congest at time 9-10 in situation 1 while in situation 2 they congest at time 6-7. With the first frame of F4 must sent before time 4 and not congesting with F3, there is no other situation for F4 to be transmitted, which means that the two flows are bound to conflict. Consequently, there is poor combinability (bandwidth utilization is zero) between them and it is not suitable for them to pass through the same edge.

Analysis of the Combinability between Two Flows
From the above examples, we have a preliminary understanding of different combinability. In this section, we analyze the combinability between two flows with different periods in more depth.
In the following analysis, we assume that the siz of all the flows are "1" and explore how many other flows can be accommodated when a certain stream exists on the edge. Specifically, as the first example shown in Section 5.1.1, four F2 can be accommodated when an F1 exists. Obviously, we can use this number to measure the combinability between two flows, or whether it is suitable for the latter flow to be transmitted via the specific edge, which we can use to select path for the flow.
Without loss of generality, we assume that flow1 with prd = p1 has been assigned to be transmitted through an edge at time 0. Then we analyze how many feasible situations for flow2 with prd = p2 are sent through the edge without congestion. We denote the time offset of the first frame of flow2 to be sent as a, and we get 0 ≤ a < p2 under the transmitting time limitation. Then the sending time for flow1 and flow2 would be n×p1 and (a + m × p2), where m and n are integers (denoted as N). If the two flows congest at an edge, the following equation can be obtained: and such that: Assuming the greatest common divisor (GCD, GCD also means the GCD of periods in this paper) of p1 and p2 is g, we have where the GCD of n0 and m0 is "1", otherwise g cannot be the GCD of p1 and p2. Additionally, a is computed as We can see that F3 and F4 congest at time 9-10 in situation 1 while in situation 2 they congest at time 6-7. With the first frame of F4 must sent before time 4 and not congesting with F3, there is no other situation for F4 to be transmitted, which means that the two flows are bound to conflict. Consequently, there is poor combinability (bandwidth utilization is zero) between them and it is not suitable for them to pass through the same edge.

Analysis of the Combinability between Two Flows
From the above examples, we have a preliminary understanding of different combinability. In this section, we analyze the combinability between two flows with different periods in more depth.
In the following analysis, we assume that the siz of all the flows are "1" and explore how many other flows can be accommodated when a certain stream exists on the edge. Specifically, as the first example shown in Section 5.1.1, four F2 can be accommodated when an F1 exists. Obviously, we can use this number to measure the combinability between two flows, or whether it is suitable for the latter flow to be transmitted via the specific edge, which we can use to select path for the flow.
Without loss of generality, we assume that flow1 with prd = p1 has been assigned to be transmitted through an edge at time 0. Then we analyze how many feasible situations for flow2 with prd = p2 are sent through the edge without congestion. We denote the time offset of the first frame of flow2 to be sent as a, and we get 0 ≤ a < p2 under the transmitting time limitation. Then the sending time for flow1 and flow2 would be n × p1 and (a + m × p2), where m and n are integers (denoted as N). If the two flows congest at an edge, the following equation can be obtained: and such that: Assuming the greatest common divisor (GCD, GCD also means the GCD of periods in this paper) of p1 and p2 is g, we have where the GCD of n 0 and m 0 is "1", otherwise g cannot be the GCD of p1 and p2. Additionally, a is computed as As n, n 0 , m and m 0 are integers, it can be concluded that a is an integral multiple of g. Then we explore the possible values for a, before which we quote the famous Bézout's identity [29]: "For nonzero integers a and b, let d be the greatest common divisor d = gcd(a,b). Then, there exist integers x and y such that: ax + by = d." Apparently, n 0 and m 0 are nonzero integers. Where the GCD of n 0 and m 0 is "1" and (n × n 0 − m × m 0 ) is a linear combination of n 0 and m 0 , we can deduce that Note that m and n can be any integer. From Equations (8) and (9), we get only when n = k × n 1 , m = k × m 1 .
By Equation (10), the value range of a (denoted as A) is Therefore, the number of possible values for a are p2/g. In other words, the number of situations in which the two flows do not congest is (p2 − p2/g). More generally, let num be the number of flows that can be accommodated, such that: From the above discussion, we find an expression that can evaluate the combinability between two flows when their siz are both "1". As for the situations when siz is an arbitrary integer, considering that siz is generally much smaller than prd in real application scenarios, we can roughly treat a flow with siz = n as n flows with siz = 1. Then, num is computed as Since prd and siz are the properties of the flow itself, GCD is the determinant of the combinability of two flows.

Combinability among Multiple Flows
The above analysis points out the combinability between two flows and the upper limit that can accommodate one flow when the other flow exists, but it is not enough to select a path for a newly added flow. A metric that can be applied to a real-world scenario, where multiple periods of flows are transmitted via a same edge, is needed.
Equation (13) indicates the maximum number of flows that can be accommodated when the other flow exists and siz are both "1". By converting this number to the specific weight (denoted as wgt) of each flow, we get As for the situation when siz is an arbitrary value, by Equation (14), we have Then the sum of wgt (denoted ad SOW) is computed as The higher the SOW, the closer to the upper limit of the flows that can be accommodated, and the more difficult it is to accommodate more flows. It means that SOW can be used to measure the scheduling bottleneck when there are flows with two kinds of periods. Similar to MSTL, we propose using the maximum SOW (denoted as MSOW) as the bottleneck of scheduling, and we get As LCM is a constant for the same set of flows, comparing Equation (2) and Equation (18), we find that there is only a difference between the two by a coefficient related to GCD. This shows that when considering the scheduling bottleneck, not only the total load on an edge must be considered, but also the combinability of flows passing through the same edge. When choosing a path for a flow, we should select a path whose MSOW is the least after adding the flow to it.
As shown in Figure 5, there are three candidate paths for F7 (prd = 6, siz = 1) to select. Table 2 shows the SOW, TSL (scheduled traffic load according to Equation (2)) and number of future flows that can be accommodated (NOF) after selecting different paths. It can be seen that the smaller the SOW, the more flows it can accommodate in the future. However, there is no such inverse relationship between TSL and NOF. It demonstrates that MSOW is more suitable than MSTL to measure the scheduling bottleneck. Notably, if there are two flows whose GCD is "1", arranging the same path for them will leave the flows unschedulable. This can be avoided by using MSOW as the metric of path selection, but not by MSTL. Consequently, trying to lower the value of SOW helps to accommodate more flows. used to measure the scheduling bottleneck when there are flows with two kinds of periods. Similar to MSTL, we propose using the maximum SOW (denoted as MSOW) as the bottleneck of scheduling, and we get = max (∑ -).
As LCM is a constant for the same set of flows, comparing Equation (2) and Equation (18), we find that there is only a difference between the two by a coefficient related to GCD. This shows that when considering the scheduling bottleneck, not only the total load on an edge must be considered, but also the combinability of flows passing through the same edge. When choosing a path for a flow, we should select a path whose MSOW is the least after adding the flow to it.
As shown in Figure 5, there are three candidate paths for F7 (prd = 6, siz = 1) to select. Table 2 shows the SOW, TSL (scheduled traffic load according to Equation (2)) and number of future flows that can be accommodated (NOF) after selecting different paths. It can be seen that the smaller the SOW, the more flows it can accommodate in the future. However, there is no such inverse relationship between TSL and NOF. It demonstrates that MSOW is more suitable than MSTL to measure the scheduling bottleneck. Notably, if there are two flows whose GCD is "1", arranging the same path for them will leave the flows unschedulable. This can be avoided by using MSOW as the metric of path selection, but not by MSTL. Consequently, trying to lower the value of SOW helps to accommodate more flows.

The Proposed Routing Algorithm
The analysis in Section 5.1 indicates that MSOW is an appropriate metric to measure the scheduling bottleneck, based on which we proposed a period-aware routing algorithm (denoted as PAR). To facilitate the understanding of our algorithm, we first explain some of the symbols used in it.

Definition 1.
We denote each flow ∈ F by a 4-tuple f ≡ (src, dst, siz, prd) in Section 3, and we add an attribute "class" (initial value is 0) to it, which is determined by our algorithm. We use

The Proposed Routing Algorithm
The analysis in Section 5.1 indicates that MSOW is an appropriate metric to measure the scheduling bottleneck, based on which we proposed a period-aware routing algorithm (denoted as PAR). To facilitate the understanding of our algorithm, we first explain some of the symbols used in it.

Definition 1.
We denote each flow ∈ F by a 4-tuple f ≡ (src, dst, siz, prd) in Section 3, and we add an attribute "class" (initial value is 0) to it, which is determined by our algorithm. We use f.prd to represent the period of the corresponding flow, and the representation methods for other attributes are similar. Definition 2. Three attributes "prd_list" (initial value is {}), "gcd" (initial value is 0) and "SOW" (initial value is 0) are added for each edge in the network. The first represents the set of periods of all flows passing through the edge (repeated elements will be merged into one element), while the second represents the greatest common divisor of all periods in the set. Additionally, "SOW" is the sum of wgt mentioned in Section 5.1.3.

Definition 5.
The function get_wgt(f) returns the wgt for the flow f using Equation (16).

Definition 6.
The function all_simple_paths(G, src, dst) returns all simple paths in the graph G from src to dst, where simple path is a path with no repeated modes.

Period-Aware Routing Algorithm
The whole procedure of the period-aware routing algorithm is presented in Algorithm 1. In general, the algorithm consists of two stages: sorting and routing. Firstly, the flow set is sorted according to the sub-algorithm, Sorting_the_flows. Additionally, the path is selected for each flow in turn. When choosing a path for a flow, all the possible paths are found in the first step (line 4). The function all_simple_paths can be easily implemented using NetworkX [30], a popular Python language package for exploring and analyzing the network. All possible paths are traversed and the best path for the flow is found using the procedure Find_best_path (line 5). Path_set.append(path*); 7: end 8: return Path_set

Sorting the Flows
In order to minimize MSOW, GCD of the flows passing through the same edge should be as large as possible. Consequently, we need to group the flows according to their impact on GCD or combinability. For instance, a flow whose GCD with every other flow is 1 has poor combinability with other flows, so the flow should use a separate path. Besides, changing the GCD of an edge after adding a flow should be avoided, otherwise SOW has to be recalculated for the previous flows. For flows that do not affect the LCM of the flows set after removal, their prd are the factor of others and they may be the GCD of an edge. We should route them earlier such that the GCD is not likely to change when the following flows are added. Therefore, we divide flows into three categories:

•
Class 0: The flows whose GCD with every other flow is 1; • Class 1: The flows do not affect LCM of the flows set after removal; • Class 2: Other flows.
Moreover, it is not likely to reduce the GCD after adding a flow with a larger period, so we sort the flows in the order of the period from small to large. The procedures of sorting the flows are presented in Algorithm 2 as follows. In step I (line 1), LCM of the flows set (lcm_old) is calculated. In step II (lines 3-4), the LCM (lcm_new) is recalculated after removing the specific flow. In step III (lines 5-11), flows are classified. If the equation in line 5 holds, it means that the flow's GCD with every other flow is 1, so it falls into class 0. While if lcm_new is equal to lcm_old, it is classified as the class 1 and the others will be class 2. The above steps will be repeated until all flows are classified. Finally, in step IV (line 14), flows are sorted from small to large according to class and period. if (lcm_new == lcm_old/f i .prd) then 6: fi.class = 0; 7: else if (lcm_new == lcm_old) 8: fi.class = 1; 9: else 10: fi.class = 2; 11: end 12: Append f i to F; 13: end 14: Fs ← sort F by f.class and f.prd from small to large 15: return Fs

Find Best Path
As a core part of our approach, the Find_best_path procedure is shown in Algorithm 3. First of all, we compute the number of edges (denoted as hops) in the path (line 1). Then MSOW of each path supposing the flow uses the path is calculated (line [3][4][5][6][7][8][9][10][11]. After that, we calculate the cost function for the path (line 12). It is computed as The purpose of our algorithm is to reduce the MSOW after all flows are routed. However, if each flow is routed to reduce MSOW as the optimization goal, a too-long path may be chosen in the front, causing each flow to add a large load to the entire network. For example, if we choose a path with eight edges for a flow, it will increase the load of this flow to eight edges. In other words, using a too-long path will reduce the bandwidth available for subsequent flows. Therefore, we introduce a user-defined positive integer K to penalize the length of the path. After the computation of the cost, we restore the state of the network to calculate the cost for the next path (lines [13][14][15][16][17]. The path with the smallest cost is taken as the optimal path after traversing all the paths. Then the network state is changed after the flow chooses the optimal path (lines 20-24). edge.SOW += wgt; 24: end 25: return path* As for calculating wgt, the procedure is shown in Algorithm 4. Note that the purpose of our previous sorting algorithm is to keep the GCD of a bridge unchanged after adding a new flow. If there is only one flow on the edge, we use the prd of the flow as GCD, which is very likely not change in future procedures. While if the GCD of an edge change, edge.SOW will have to be recalculated for the previous flows (lines [4][5][6][7]. Although recalculating the value will increase the complexity of the algorithm, the overall complexity of the algorithm is not high because of the low probability of this happening. Moreover, if the GCD of an edge is equal to 1, the value of wgt is set to a large enough integer such that the path will not be selected. This can avoid unschedulable situations due to GCD being 1.

Algorithm 3 Find best path
The four algorithms we proposed have been introduced, and then we analyze the complexity of these algorithms. We use |F|, |N| and |E| to denote the number of flows, nodes and edges in the graph, respectively. All flows are traversed in Algorithm 2, so the complexity of Algorithm 2 is O(|F|). The complexity of Algorithm 4 is O(1) since there is no loop in it. There are two layers of loops in Algorithm 3, which are the traversal of the number of paths and the traversal of the edges in the path. We roughly estimate that the number of paths for each flow is 2|E| and the number of edges in each path is |E|/2. Therefore, the complexity of the Algorithm 3 is O(|E| 2 ). As for Algorithm1, it traverses all flows, finds all possible paths of each flow, and then selects the optimal path from these paths. In the procedure all_simple_paths, the complexity of finding a single path for a flow is O(|N|+|E|) and the complexity of finding all the paths is O((|N|+|E|)|E|/2). Since |E| > |N| in our model, O((|N|+|E|)|E|/2) < O(|E| 2 ). So we can conclude that the complexity of the Algorithm 1 is O(|F||E| 2 ).

Experiments and Evaluation
In this section, the evaluation is conducted in terms of the percentage of successfully scheduled flow sets compared to two baseline algorithms. The first algorithm is shortest path routing (denoted as SPR), which is most commonly used in practical applications. The second is a recently published load balanced routing (denoted as LBR) proposed in literature [3]. In order to exclude the influence of irrelevant factors, we evaluated the performance from three dimensions: load, topology and flow type. Based on the method proposed in literatures [6], we built an ILP-based no-wait scheduling platform to evaluate the success rate of scheduling.

Experiment Setup
The routing algorithms and scheduling platform were implemented in a Python-based framework, where we created a network and executed the routing algorithm using the NetworkX API and solved the ILP-based scheduling with the Gurobi Optimizer API. The experiments were performed on a PC with an Intel Core i5-8265U processor running at 1.8 GHz and 8 GB RAM.
The topologies used in this article are quoted from literature [6,8]. As is shown in Figure 6a, the topology quoted from literature [8] is a partial mesh topology including 12 end stations and 12 switches. While there are 4 end stations and 6 switches in the topology quoted from literature [6], which have a slightly higher connectivity.
Two groups of candidate flows are presented in Tables 3 and 4. In the first group, the period of each flow is a multiple of 10, which means there is high combinability between every two flows. While there is a big difference in the combinability between two flows and even the GCD may be 1 in the second group. The probability of being selected for F1 is relatively low because it has low combinability with other flows. When generating a set of flows, we first specified a group, and then randomly selected flows from it according to their respective probabilities. Besides, the source and destination of each flow were randomly selected. For a single test case, we generated 100 flow sets with the same number of flows. Two groups of candidate flows are presented in Tables 3 and 4. In the first group, the period of each flow is a multiple of 10, which means there is high combinability between every two flows. While there is a big difference in the combinability between two flows and even the GCD may be 1 in the second group. The probability of being selected for F1 is relatively low because it has low combinability with other flows. When generating a set of flows, we first specified a group, and then randomly selected flows from it according to their respective probabilities. Besides, the source and destination of each flow were randomly selected. For a single test case, we generated 100 flow sets with the same number of flows. 1 The probability to be selected, similarly below.

Evaluation Results
A wide range of test cases was used to evaluate our algorithm. We evaluated the performance of the proposed algorithm (PAR) and two baseline algorithms (SPR and LBR) by the scheduling result computed by Gurobi Optimizer. The time limit for scheduling solution was set to 1 min and solution number limit was set to 1, so there were three possible solver outcomes: • Solved: One sub-optimal schedule found (optimal solution is not pursued in this paper). • Timeout: Time limit reached without a feasible schedule. • Infeasible: Cannot be scheduled with the given flows and routes.

Load Dependency
Firstly, three test cases were performed to show the schedulability of the algorithms under different network loads. They used the topology from Figure 6a (topology 1) and flows from Table 3 (flow group 1). We set K = 0.4 for PAR here and the result is presented in Figure 7. This figure shows that PAR achieved higher schedulability under different network loads.
• Solved: One sub-optimal schedule found (optimal solution is not pursued in this paper). • Timeout: Time limit reached without a feasible schedule. • Infeasible: Cannot be scheduled with the given flows and routes.

Load Dependency
Firstly, three test cases were performed to show the schedulability of the algorithms under different network loads. They used the topology from Figure 6a (topology 1) and flows from Table 3 (flow group 1). We set K = 0.4 for PAR here and the result is presented in Figure 7. This figure shows that PAR achieved higher schedulability under different network loads.  Specifically, when there are only 20 flows in the network, flows are not likely to congest, so the advantage of PAR is not obvious. While as the number of the flows increase, the probability of a highly utilized edge causing the infeasibility is growing. In this situation, the benefit of well-designed algorithms to reduce the scheduling bottlenecks is reflected, and the improvement of schedulability is also more obvious. Although the combinability between flows is similar, our algorithm can better reduce the bottleneck than LBR and improve the schedulability by considering the combinability. This suggests that more flows can be accommodated using our algorithm. Specifically, when there are only 20 flows in the network, flows are not likely to congest, so the advantage of PAR is not obvious. While as the number of the flows increase, the probability of a highly utilized edge causing the infeasibility is growing. In this situation, the benefit of well-designed algorithms to reduce the scheduling bottlenecks is reflected, and the improvement of schedulability is also more obvious. Although the combinability between flows is similar, our algorithm can better reduce the bottleneck than LBR and improve the schedulability by considering the combinability. This suggests that more flows can be accommodated using our algorithm.

Topology Dependency
Secondly, we analyze the schedulability of three algorithms using the topology from Figure 6b to exclude the impact of a specific topology. Flow type is the same as Section 5.2.1 while the flow number is smaller since we used a smaller topology. We set K = 0.3 for our algorithm and the result is shown in Figure 8. As shown below, the schedulability of the three algorithms as the number of flows increases is similar to the result presented in Figure 7. These test cases demonstrate that PAR is useful in different topologies. It is worth noting that when the flow number is 20, the number of the timeout sets for PAR and LBR is significantly higher than that of SPR. This is because some edges may not be used for SPR and the complexity of ILP-based scheduling is reduced. Figure 6b to exclude the impact of a specific topology. Flow type is the same as Section 5.2.1 while the flow number is smaller since we used a smaller topology. We set K = 0.3 for our algorithm and the result is shown in Figure 8. As shown below, the schedulability of the three algorithms as the number of flows increases is similar to the result presented in Figure 7. These test cases demonstrate that PAR is useful in different topologies. It is worth noting that when the flow number is 20, the number of the timeout sets for PAR and LBR is significantly higher than that of SPR. This is because some edges may not be used for SPR and the complexity of ILP-based scheduling is reduced.  Figure 8. Percentage distribution of solved, timeout and infeasible sets for PAR, LBR and SPR for different numbers of flows using topology 2 and flow group 1.

Flow Type Dependency
Finally, we illustrate the advantages of our algorithm in terms of flow type. Topology 1 and flow group 2 were used in these test cases and we set K = 0.4. From Figure 9, we can see that the results of experiment are completely different from the above two. This figure shows that the success rate of LBR and SPR is almost the same while PAR has made a big improvement in this aspect. This is due to the big difference in the combinability between the flows used in these cases. For example, if F1 and F2 of the group 2 are arranged to pass through the same edge, it will directly lead to infeasible situations. In this case, how to make the flow with poor combinability follow disjointed paths, rather than how to optimize the load distribution, becomes the key to the success of scheduling. As LBR only considers load distribution and ignores the combinability between flows, it does not achieve better results than SPR.

Flow Type Dependency
Finally, we illustrate the advantages of our algorithm in terms of flow type. Topology 1 and flow group 2 were used in these test cases and we set K = 0.4. From Figure 9, we can see that the results of experiment are completely different from the above two. This figure shows that the success rate of LBR and SPR is almost the same while PAR has made a big improvement in this aspect. This is due to the big difference in the combinability between the flows used in these cases. For example, if F1 and F2 of the group 2 are arranged to pass through the same edge, it will directly lead to infeasible situations. In this case, how to make the flow with poor combinability follow disjointed paths, rather than how to optimize the load distribution, becomes the key to the success of scheduling. As LBR only considers load distribution and ignores the combinability between flows, it does not achieve better results than SPR.
In contrast, our algorithm mainly considers combinability in path selection, so flows whose GCD is 1 will not share the same edge and the schedulability is doubled. This shows that the proposed algorithm is more adaptable to flows with a variety of periods. It is of practical value under the premise that there are more and more sensors in the system and the types of flows are becoming more and more complex.  Figure 9. Percentage distribution of solved, timeout and infeasible sets for PAR, LBR and SPR for different numbers of flows using topology 1 and flow group 2.
In contrast, our algorithm mainly considers combinability in path selection, so flows whose GCD is 1 will not share the same edge and the schedulability is doubled. This shows that the proposed algorithm is more adaptable to flows with a variety of periods. It is of practical value under the premise that there are more and more sensors in the system and the types of flows are becoming more and more complex.

Conclusions
In this article, a novel routing heuristic for time-triggered flows was introduced. We discovered that periodic flows use decentralized bandwidth, and put forward the concept of combinability to express the ability of different streams to use dispersed bandwidth. By analyzing the combinability of two or more periodic flows, we proposed a metric to measure the scheduling bottleneck. Moreover, we found that there is only one coefficient related to the greatest common divisor between our metric and the existing methods, and the great common divisor is an important factor to measure the combinability between flows. We implemented test cases to evaluate the schedulability of our algorithm compared to shortest path routing and a recently proposed load balanced routing. The results were analyzed with respect to three dimensions: load, topology and flow type. Additionally, they show that our method can achieve better schedulability regardless of the load and topology in the case of high combinability between flows, which means that more time-triggered flows can be accommodated. As for the circumstances when combinability between flows is poor, our algorithm doubled the schedulability compared to the existing method, while load balanced routing gained no improvement compared with the shortest path routing. In the case of more and more complex flows in the network, better flow type

Conclusions
In this article, a novel routing heuristic for time-triggered flows was introduced. We discovered that periodic flows use decentralized bandwidth, and put forward the concept of combinability to express the ability of different streams to use dispersed bandwidth. By analyzing the combinability of two or more periodic flows, we proposed a metric to measure the scheduling bottleneck. Moreover, we found that there is only one coefficient related to the greatest common divisor between our metric and the existing methods, and the great common divisor is an important factor to measure the combinability between flows. We implemented test cases to evaluate the schedulability of our algorithm compared to shortest path routing and a recently proposed load balanced routing. The results were analyzed with respect to three dimensions: load, topology and flow type. Additionally, they show that our method can achieve better schedulability regardless of the load and topology in the case of high combinability between flows, which means that more time-triggered flows can be accommodated. As for the circumstances when combinability between flows is poor, our algorithm doubled the schedulability compared to the existing method, while load balanced routing gained no improvement compared with the shortest path routing. In the case of more and more complex flows in the network, better flow type compatibility will have great practical value.
Still, shortcomings exist in our algorithm. As an empirical value in our method, K should be determined according to the characteristics of the flow and the scale of the topology, and the selection of K will greatly affect the performance of the algorithm. In the future, we would develop heuristics considering the scheduling bottleneck globally rather than thinking about it flow by flow, which can no longer rely on an empirical parameter.