Evaluation of the Effects of Mobile Smart Object to Boost IoT Network Synchronization

This paper deals with the synchronization of Mobile Smart Objects (MSOs). Today, this scenario is becoming typical in Industrial IoT applications due to the plethora of MSOs available as robots, drones and wearables, equipped by sensors making them measurement instruments cooperating in distributed measurement systems. In this context, the synchronization accuracy is directly tied with the accuracy of the performed measurements. In hierarchical synchronization approaches, the presence of an MSO makes the network topology time varying, and this could prevent the synchronization of the whole network. Peer to peer approaches do not need node hierarchy to synchronize but could not converge to a common sense of time. To overcome these challenges, this paper proposes a consensus-based approach for which the convergence to a common sense of time is here demonstrated. The proposal deploys the MSO to bring the common sense of time from an SO to another, establishing new paths among SOs. The new paths are temporary and depend on the MSO’s route. In the paper, the influence of the MSO’s route on the synchronization accuracy σ and the time interval to synchronize all the SOs ∆TIS is investigated, also. The mathematical proof, the simulations and the experimental tests confirm that the MSO can reduce both the values of σ and ∆TIS, because the new connections introduced by the MSO can boost the exchange of information among SOs. Consequently, the criteria to a priori select the route ameliorating σ and ∆TIS values are proposed.

In recent years, low-power wireless SOs have gained popularity [18]. They interact to form large networks such as Machine to Machine (M2M) networks and Wearable Computing. In these contexts, the major challenge is to guarantee robust communication between elements while keeping the whole network energy efficient. The time synchronization permits them to coordinate communication slot for the SOs [4], implementing a time division multiple access communication protocols, i.e., the SO has to forward messages to its neighbors in only a short wakeup period.
The time synchronization of the measurements requires that all the network SOs share the common sense of time. The synchronization accuracy σ among SOs affects the efficacy of the data retrieval, while the time interval required to synchronize all the SOs (∆ TIS ), i.e., the time interval starting with the execution of the synchronization algorithm and ending The correction parametersα i (t), andô i (t) are evaluated by each SO on the basis of synchronization messages (SynMsg) exchanged among the communicating SOs by the radio section.
The radio section is in charge of receive SynMsgs from communicating SOs and to send, in broadcast modality, the SynMsg each t k = kT P , k ∈ N + , where T P is the synchronization time period. The message SynMsg ji , sent at t k by SO#j and received by SO#i, includes: id j ,α j (t k ),ô j (t k ), τ j (t k ) , where id j is the SO identifier that is unique for each SO.
At the reception of the SynMsg ji , the radio section communicates this event to the µ-processor that starts the synchronization procedure of the block scheme of Figure 1.  In Figure 1, τ ij is the number of clock impulse of SO#i at the reception of SynMsg ij . The variables τ old j and τ old ij refer to the previous reception of SynMsg ij , ρ v and ρ o are design parameters (memory factors) that can be set in the range [0, 1], and t + k indicates the upgraded value of the time dependent variablesα i (t), andô i (t), respectively [31]. In particular, if ρ v = 0 and ρ o = 0, the node will correct itsα i (t) andô i (t) on the basis of the time values of its neighbors only, and vice-versa: if ρ v = 0.9 and ρ o = 0.9, the upgrade of α i (t) andô i (t) will mainly depend on its own value.
The MSO is used to bring the sense of the time from one SO to another. If it executes the synchronization procedure for the first time, the SOs ignore its message. This condition is checked by theα j (t) andô j (t) values in the message received from the MSO and verified that they are equal to the initialization values.
In order to speed up the convergence of the MSO to the common sense of time of the network and to avoid that it could perturb the network synchronization, at the first synchronization time instant (t k , with k = 1), the MSO takes the sense of time of the first received message. This condition is checked by the MSO by its ownα i (t) and o i (t) values and verified that they are equal to the initialization values. At the end of the first synchronization time instant (t k + , with k = 1), MSO will haveτ i (t) bounded by the maximum and minimum values of theτ j (t) of the SOs in its BR, because it: (i) is in the same neighborhood of the visited SO, (ii) receives the same synchronization messages, and (iii) executes the same upgrade ofα i (t) andô i (t). In the successive synchronization time instants (t k , with k > 1), MSO will haveτ i (t) bounded by the maximum and minimum values of theτ j (t) of the SOs.
The number of connections in the paths among SOs influences the time with whom the information of frequency and offset of each SO clock is shared among all SOs. In particular, the fully connected topology represents the best case, because each SO is directly connected to all others. The worst case is if a nonconnected SO is present. The isolated SO cannot be synchronized, because no information is shared with it.
Because the MSO moves, it increases the number of paths among SOs, creating new connections, but these connections vary in time according with the MSO route.
In the following Section 3, the conditions making the synchronization procedure success are investigated, and the effect of the MSO route on σ and ∆ TIS is analyzed.

Convergence Assessment
An effect that must be taken into account by introducing the MSO in an IoT wireless network synchronized by the consensus-based technique of Section 2 concerns the convergence assessment to a common sense of time. The MSO spreads its sense of time in the network and potentially can make the consensus process fail [26]. A formal proof that the synchronization procedure of the Section 2 makes the consensus process successful is to verify the convergence condition of the Theorem#1 of [26]. Theorem#1 assesses that the consensus is stable, i.e., the SOs converge to a common sense of time, if the variable estimated by MSO is always bounded by the maximum and minimum estimation of the corresponding variables of each SO. At the starting of the synchronization procedure, the SOs ignore the MSO's message, and the MSO takes the sense of time of the first received message. At the successive synchronization steps, the MSO and the SOs will execute the same synchronization procedure. Therefore, the MSO'sτ i (t) is always bounded in the maximum and minimum values of the SOs, satisfying the Theorem#1.

Reduction of σ Value
Another effect, important for measurement purposes, is the reduction of the synchronization accuracy σ. This effect is not considered in [26] and in the theoretical approaches, since they assume that, after the convergence, the difference between the estimated time values of any couple of SOs is zero. This condition cannot be achieved in practical cases, due to the nonidealities of the SO's hardware components. In [31], the effect of the nonide-alities is modeled by introducing the bounded time-varying disturbance vector d(t), with maximum value d max , that depends on the hardware characteristics of the SO clocks as the clock noise, the quantization error introduced by the limited resolution of the clocks, and so on. The clock hardware characteristics determine the lower bound of the achievable synchronization accuracy. The SOs and the MSO update the hardware clock value each T seconds, according to (1) at t s+1 = t s + T: where s is positive integer, α is the clock frequency nominal value, and d i (t s ) is the i-th element of d(t) such that d i (t s ) ≤ d max . As a consequence, at t s+m = (s + m)T, m positive integer (without executing the synchronization procedure) it is: where At t s = t k and m = m synch = T P /T, the synchronization time occurs. By considering, for sake of simplicity, the case where only offset synchronization is performed, it is assumed According to (2) and the upgrade rule ofô i , Figure 2, it is: By considering a synchronous communication model among SOs and MSO, from (5), it is possible to evaluate the update of the software clock time of each SO and of the MSO.
, L is the Laplacian matrix depending on the connections of the communication graph including the MSO, and ε = 1 − ρ 0 . In Equation (6),τ i t + k is the sum of all the contributes, evaluated according to Equation (5), of the software clocks of the SOs and MSO in the BR.
From Equation (4), because no communication is performed between two consecutive synchronization time instants t k and t k+1 , the software clock values are: where By Equation (6), Equation (7) becomes: According to graph theory [32,33], if the network graph is connected, the matrix I + εL has only one eigenvalue with value equal to 1, λ 1 = 1, while all the others are strictly lower than one (|λ| i < 1, i = 2, . . . , (Nc + 1)). In addition, the eigenvector associated to λ 1 is the unit vector v 1 with dimension Nc + 1.
To evaluate the influence of the MSO on the synchronization accuracy, it is defined the synchronization delay between SO#i and SO#j as y ij (k) = C ij τ(t k ), where C ij = e i − e j T and e i , e j are the i-th and j-th vectors of the canonical basis. Since C ij v 1 = 0, i.e., an object is with delay zero respect to itself, the v 1 does not contribute to the synchronization delay, and, consequently, on the synchronization accuracy. Moreover, the eigenvalues λ 1 has no influence on the synchronization delay. Since all the other eigenvalues are strictly lower than 1, the maximum value of the synchronization delay is bounded with respect to d max , as follows: C i,j (I + εL) k e r is the 1-norm of the impulsive response matrix between the disturbance vector and the output y ij [33]. Equation (9) shows that the maximum value of the synchronization delays, and consequently, the synchronization accuracy is linked in a proportional way with the parameter l 1 that is influenced by the second largest eigenvalue of the matrix, which depends upon the connectivity of the graph. The more the graph is connected, the lower is the l 1 value and, therefore, the influence of the disturbances on the synchronization accuracy. The MSO, by communicating with the SOs in its BR, creates new connections and, consequently, new path among SOs, increasing the connectivity of the graph.

Reduction of ∆ TIS Value
Further effect of MSO concerns with the reduction of ∆ TIS . Indeed, the lower is y ij (k) ∞ , the earlier the synchronization delay between any couple of SOs becomes lower of the admissible delay range. Consequently, more the graph is connected, the lowest is the ∆ TIS value. This result is also confirmed by Theorem#3 of [26], assessing that, if the MSO is used, the ∆ TIS is reduced with respect to the case of a static network, i.e., only SOs. This theorem is based on the comparison of the second eigenvalue of the matrix I + εL in the two-abovementioned cases. In particular, it is proved that, with the MSO, the second eigenvalue of the I + εL is higher with respect to the case the MSO is not used, then the second eigenvector is lower, i.e., the network converges faster.
Nevertheless, the quantitative analysis of the ∆ TIS reduction is still an open challenge [26].

Evaluation of the Effect of MSO on the IoT Network Synchronization
In order to evaluate the effect of the MSO on the synchronization, it is necessary to evaluate not only the increasing number of connections due to the newly introduced paths but also how such connections are spatially distributed. Indeed, in the extreme case that a SO is not linked to the network, if it is periodically visited by the MSO, as in [34], the To evaluate the effect of the MSO route on the network connections, a lattice network topology is considered. In this topology, all the SOs (except the ones on the boundary) have the same number of connections. In this case, the MSO effect on the connections is evaluated by two statistical parameters: the mean and standard deviation of the minimum number of connections among any couple of nodes on the observing time window. In the following, node refers to both SOs and MSO. In particular, because the nodes exchange the SynMsg each T P , the effect of the MSO on the connections among nodes is evaluated at each synchronization time t k . By considering at t k the network connection's adjacency matrix A(t k ), with dimension (Nc + 1) × (Nc + 1), the average minimum number of connection among nodes µ con , activated from the start of the synchronization procedure up to t k , is: The standard deviation is: The element a ij (t k ) of the matrix A(t k ) is the number of connections between node#i and node#j at t k as a consequence, the element in position i,j of the matrix the number of connections between node#i and node#j available from the start of the synchronization procedure up to t k . The operator DST( ) of Equations (10) and (11) receives as input the adjacency matrix and returns the matrix where the element dst ij is the minimum distance between node#i and node#j. The operator mean( ) of Equation (10)  Because the synchronization parameters ∆ TIS and σ depend on (i) the network topology, (ii) the number of SOs, and (iii) the MSO's routes, numerical tests are designed to assess the theory in Section 3. In this way, the relationships among ∆ TIS , σ µ con (t k ), and σ con (t k ) [28] are highlighted. These relationships allow the establishment of the more convenient criteria in the design of the MSO route.
In the tests, the MSO visits the SOs in the predefined route. Since the MSO propagates the common sense of time at t k , it estimatesτ i t + k according to the synchronization parameters of the last visited SOs (see Section 2). It is assumed that, at t k , the MSO (dashed node in Figure 2) is directly connected with the visited SO and the SOs are directly connected to the visited SO. Both MSO and SOs communicate in wireless broadcasting mode.
In order to easily understand the effect of the MSO on the SO connections, a square lattice topology with different values of Nc is considered.
The T clk andτ i (t 0 ) values are selected, respectively, according to the hardware realization of the clock typically equipping commercial MSO and SO [30] and the time required to power-on all the SOs and the MSO. In particular, T clk is selected equal to 1 ms with variation in the range [0.999980, 1.000020] T clk ,τ i (t 0 ) in the range [0, 30] × 10 7 T clk . The observation time interval of each test is set equal to 10,000 s, and each node collects its own time interval each T = 2 σ and sends a SynMsg each T P = 10 s.
To evaluate the effect of the MSO, different MSO's routes are considered. Among all these routes, in the following, particular interest is devoted to two different sets.
One set that includes straight-line routes, the main diagonal (Route#1-3 of Figure 2), is selected, because it splits symmetrically the network. The selection of the second diagonal or the central row or central column would not give further information, since they are a rotation of the main diagonal respect to the center of the matrix. Therefore, the results referred to these routes are not reported. Another set includes the routes selected with the following two criteria (Route#4-6 of Figure 2): (i) peripheral nodes of the network are visited, and (ii) the number of the node connections at each t k is maximized.
Random topologies of the network are not considered in this research, because they make difficult to understand the relationship between the MSO's route and ∆ TIS , since ∆ TIS depends not only on the number of the visited nodes but also on their position in the network and the order with which MSO visits the nodes.
Let N i,j , i, j = 1, . . . , n, the node of the square lattice topology, where n = sqrt(Nc). Imposing that at time t k , MSO is on N i,j , the following routes are selected: V = 1; n; 2; n − 1; . . . ; n 2 ; n 2 + 1; n 2 ; n 2 + 1; n 2 ; . . . ; n − 1; 2; n , k ∈ N + • Route#3: . . . ; n 2 − 2; n 2 + 2; n 2 ; n 2 + 2; n 2 − 2 . . . ; n − 1 n 2 mod(2)= 0 V = 2; n − 1; . . . ; n 2 ; n 2 ; n 2 ; . . . ; n − 1 To evaluate ∆ TIS , the admissible delay range equal to 100 T clk, is imposed; σ is evaluated on the last 1000 s of the observation time interval. This time interval is selected on the basis of preliminary tests to guarantee that σ is evaluated when the common sense of time among the SOs is achieved. Figure 3 shows the percentage improvement of ∆ TIS and σ and percentage variation of µ con and σ con , with respect to the case of static topology.  By analyzing the numerical data, ∆TIS and σ are reduced of the 35% and 36%, respectively. The numerical results show that, in all cases, the MSO increases the number of connections among SOs. Lower σ con % corresponds with higher improvement in σ. Concerning ∆ TIS , according to the theory, the MSO improves its values, but the quantitative analysis is still an open challenge, and it will be studied in future research. Figure 4 shows the trend of the delay between node#25 taken as a reference and all the others in the case Figure 4a  The numerical results show that, in all cases, the MSO increases the number of connections among SOs. Lower σcon% corresponds with higher improvement in σ. Concerning ∆TIS, according to the theory, the MSO improves its values, but the quantitative analysis is still an open challenge, and it will be studied in future research. Figure 4 shows the trend of the delay between node#25 taken as a reference and all the others in the case Figure 4a   By analyzing the numerical data, ∆TIS and σ are reduced of the 35% and 36%, respectively. For sake of completeness, a random movement of the MSO is considered, and Figure  5 shows the trend of the delay among nodes in the case Nc = 49 and (a) static topology, and (b) MSO is randomly moving. In particular, the position of the MSO is established, time by time, by extracting an integer number in the range 1-49. As shown in Figure 5, as foreseen from theory, this random path gives better results with respect the other paths.
In particular, ∆TIS is equal to 1400 s, and σ is equal to 10 Tclk. Such result is justified by the fact that the probability that the MSO links far nodes is increased. By analyzing the numerical data, ∆ TIS and σ are reduced of the 35% and 36%, respectively.
For sake of completeness, a random movement of the MSO is considered, and Figure 5 shows the trend of the delay among nodes in the case Nc = 49 and (a) static topology, and (b) MSO is randomly moving. In particular, the position of the MSO is established, time by time, by extracting an integer number in the range 1-49. As shown in Figure 5, as foreseen from theory, this random path gives better results with respect the other paths. In particular, ∆ TIS is equal to 1400 s, and σ is equal to 10 Tclk. Such result is justified by the fact that the probability that the MSO links far nodes is increased.

Experimental Tests and Discussion
Experimental tests were carried out to evaluate the effect of MSO in a real scenario.
The experimental testbed consisted of 23 Crossbow TelosB nodes, 30 xm1000 nodes and a PC. In particular, 21 TelosB nodes (indexed from 1 to 21) and all the xm1000 nodes were used to implement the synchronization algorithm. The TelosB node (indexed as 101) was plugged into the PC and used as ZigBee/USB protocol converter. The MSO is implemented in a xm1000 node indexed as 50. The TelosB node (indexed as 102) was used to trigger the 50 nodes to send the service messages containing their identification number  Figure   6 shows the actual realization of the testbed.

Experimental Tests and Discussion
Experimental tests were carried out to evaluate the effect of MSO in a real scenario.
The experimental testbed consisted of 23 Crossbow TelosB nodes, 30 xm1000 nodes and a PC. In particular, 21 TelosB nodes (indexed from 1 to 21) and all the xm1000 nodes were used to implement the synchronization algorithm. The TelosB node (indexed as 101) was plugged into the PC and used as ZigBee/USB protocol converter. The MSO is implemented in a xm1000 node indexed as 50. The TelosB node (indexed as 102) was used to trigger the 50 nodes to send the service messages containing their identification number id i , the software clock frequencyα i (t), the software clock offsetô i (t), and the hardware clock τ i value to the ZigBee/USB protocol converter (and, in turn, to the PC) at sampling period T = 2 s. The data collected by the PC was used to evaluate the synchronization accuracy in terms of discrepancy among the software clocksτ i (t) for all i ∈ [1,50]. Figure 6 shows the actual realization of the testbed.
In the TelosB [32] and xm1000 [35], the microcontroller is a TIMSP430 featuring a Digitally Controlled Oscillator (DCO) running at 8 MHz with a clock period T DCO = 0.125 µs and 10 kbyte of RAM.
Furthermore, it also features an External Crystal Oscillator (ECO) running at 32,768 Hz. Regarding the radio equipment, the nodes feature the IEEE 802.15.4/ZigBee-compliant module equipped with a RF transceiver operating within the range 2.4000-2.4835 GHz, compatible with ISM band, which allows a data rate of 250 kbps. An important feature of the radio chip CC2420 is the MAC-layer timestamp capability. This allows each node to read the local clock at the beginning of the transmission or reception of the start frame delimiter (SFD) of a message. This mechanism strongly reduces the random delays introduced by the transmission and the readings of the synchronization messages. Indeed, this mechanism was used in [36] to support their major assumption, i.e., communication delays are negligible with respect to T clk . The adoption of the precision tag "TMilli" giving a millisecond resolution of T clk was not cosmetic, as there are several hardware platforms, such as the Memsic's IRIS mote [37], for which this represents the only option available. Thus, this reflects a realistic operating condition in several application contexts.
mented in a xm1000 node indexed as 50. The TelosB node (indexed as 102) was used to trigger the 50 nodes to send the service messages containing their identification number idi, the software clock frequency ( ) i t α , the software clock offset ( ) i o t , and the hardware clock τi value to the ZigBee/USB protocol converter (and, in turn, to the PC) at sampling period T = 2 s. The data collected by the PC was used to evaluate the synchronization accuracy in terms of discrepancy among the software clocks ˆ( ) i t τ for all i ∈ [1,50]. Figure   6 shows the actual realization of the testbed. Figure 6. Experimental testbed. Figure 6. Experimental testbed.
TelosB and xm1000 motes have been programmed by TinyOS, an open-source operating system specifically designed for WSN [38].
In order to allow SOs and MSO to periodically send their clock time values to Node#101 and then to the PC, all them must be in the Node#101 BR [28,31,36]. To this aim, the nodes were deployed in a region of approximately one square meter. The high spatial density of nodes was chosen to evaluate the effectiveness of the proposed algorithm in a saturated spectrum network. For each node i-th, the neighborhood was predefined in order to achieve a lattice topology.
As a consequence, the lattice topology and the movement of the MSO are realized by defining in each SOs and in the MSO a time variant Boolean vector ND i (t k ) of the neighbors with i = 1 . . . (Nc + 1). If ND i (t k )[j] = 1, the j-th node, at time t k , is in the neighborhood of the i-th node. The MSO routes are implemented by changing the update rules of ND i (t k ).
The following parameters were used: ρ v = 0.9 and ρ o = 0.9. These are memory factors [36], and then their values influence the speed with which the whole network converges to a common sense of time. In order to compare numerical and experimental tests, ρ v and ρ o are the same for all the tests. Figure 7 shows the trend of the delay between SO#25 as reference and all other nodes of the network in the case (a) static topology, and (b) MSO moves according to the route#4.
By comparing Figure 7 with Figure 4, the same trend of the delay among SOs can be noted. The reduction of the delay among nodes shows the robustness of the consensus to the packet loss, as proved in [28,36].
By analyzing the experimental data, ∆ TIS and σ are reduced of the 28% and 38%, respectively.
The following parameters were used: ρv = 0.9 and ρo = 0.9. These are memory factors [36], and then their values influence the speed with which the whole network converges to a common sense of time. In order to compare numerical and experimental tests, ρv and ρo are the same for all the tests. Figure 7 shows the trend of the delay between SO#25 as reference and all other nodes of the network in the case (a) static topology, and (b) MSO moves according to the route#4. By comparing Figure 7 with Figure 4, the same trend of the delay among SOs can be noted. The reduction of the delay among nodes shows the robustness of the consensus to the packet loss, as proved in [28,36].

Conclusions
To overcome the convergence challenge to a common sense of time in the case of a Mobile Smart Object (MSO), in the paper, a novel synchronization algorithm based on consensus is proposed. This challenge is introduced by the fact that an MSO makes the network topology time varying, provoking the continuous evaluation of the logical tree to spread the sense of time. The consensus does not need logical tree, and, in this paper, the proof of the convergence was given for the proposed algorithm. Theoretical results, here presented, have highlighted that the MSO boosts the time synchronization of the network. This result is logically justified by the fact that the MSO brings the common sense of time among SOs by increasing their connections according to its path. The effects of the MSO's route on the statistical parameters describing the number of connections during the time and then on the values of σ and the ∆ TIS is also investigated both by numerical and experimental tests that give compatible results and confirm the theoretical findings. Consequently, the criteria to select the route reducing σ and the ∆ TIS are given. Ongoing activities are concerned with the investigation of the effect of the distance among nodes, so as of the speed of the MSO in reaching SOs, on the ameliorant of the σ and the ∆ TIS values. Moreover, the case of multiple MSOs cooperating in bring the common sense of the time to the SOs will be also taken into account.