EE-MPTCP: An Energy-Efficient Multipath TCP Scheduler for IoT-Based Power Grid Monitoring Systems

Abstract

The Internet-of-Things (IoT) based monitoring system has significantly promoted the intelligence and automation of power grids. The inspection robots and wireless sensors used in the monitoring system usually have multiple network interfaces to achieve high throughput and reliability transmission. The concurrent usage of these available interfaces with Multipath TCP (MPTCP) can enhance the quality of service of the communications. However, traditional MPTCP scheduling algorithms may bring about data disorder and even buffer blocking, which severely affects the transmission performance of MPTCP. And the common MPTCP improvement mechanisms for IoT lack sufficient attention to energy consumption, which is important for the battery-limited wireless sensors. With the aim to promote conservative energy without loss of throughput, this paper develops an integrated multipath scheduler for energy consumption optimization named energy-efficient MPTCP (EE-MPTCP). EE-MPTCP first constructs a target optimization function which considers both network throughput and energy consumption. Then, based on the proposed MPTCP transmission model and existing energy efficiency model, the network throughput and energy consumption of each path can be estimated. Finally, a heuristic scheduling algorithm is proposed to find a suitable set of paths for each application. As confirmed by experiments based on Linux testbed as well as the NS3 simulation platform, the proposed scheduler can shorten the average completion time and reduce the energy consumption by up to 79.9% and 79.2%, respectively.

1. Introduction

Power grids play a significant role in the daily life and national economy. Many modern intelligent devices such as inspection robots, Phasor Measurement Units (PMU) and the Advanced Metering Infrastructure (AMI) are utilized to monitor the system’s real time states [1]. A large amount of monitoring data including high-resolution photos and videos should be transmitted to the remote monitoring centers [2]. Intergraded with artificial intelligence, autonomous control and human interaction, the power grids become more safe and efficient. The robots as well the IoT devices, like sensors and actuators, are usually battery-supplied, wireless, multiple interfaces enabled, and heterogeneous in structure and functionality [3]. How to ensure efficient and reliable communication in such IoT-based monitoring systems for power substations is of key critical.

To achieve this aim, some researchers adopt the emerging MPTCP technology to simultaneously schedule application data of these intelligent multi-homed devices through multiple available paths. MPTCP is an extension of regular TCP and allows data streams to be delivered across multiple simultaneous connections and consequently paths. It has been widely applied in practical applications, like Apple’s iOS 7 commercial operating system and the Android system smartphone GALAXY S6 of South Korean Samsung [4]. In addition, many academic studies have shown that MPTCP can achieve improved throughput performance and reliable communication in IoT [5].

MPTCP may suffer performance degradation caused by out-of-order (OFO) data when the paths are heterogeneous as a packet with a higher sequence number may arrive at the receiver earlier than that with a lower sequence number. When the receiver buffer is limited, the OFO packets may eventually occupy the entire buffer, stalling the sender’s transmission, resulting in the Head-of-Line Blocking (HoL-Blocking) problem [4]. This scenario is common in IoT-based monitor systems as the sensors often have different wireless interfaces. Many improvements have been proposed. However, existing studies often ignore the energy consumption, which has an important fact on the battery-powered mobile wireless sensors in the IoT domain. Thus, taking a further investigation on a green transmission strategy is needed for the IoT-based monitor systems for power grids with high resource demanded like computing and networking.

The paper focuses on the MPTCP scheduling and presents a green MPTCP scheduler named EE-MPTCP (energy-efficient multipath TCP) based on mathematical optimization, which aims to greatly improve system energy efficiency, increase the effective throughput and thus guarantee the QoS of the IoT based power grid monitoring system. The main contributions are as follows.

• The proposed EE-MPTCP scheduler constructs a target optimization function whose value increases with the increase of network throughput and decrease with energy consumption.
• The markov chain based MPTCP transmission model, which constructs the change of congestion window under packet loss rate and congestion control strategy in each Round-Trip Time (RTT) cycle, is adopted to estimate the average throughput.
• A heuristic scheduling algorithm is proposed based on two-level sorting to find the optimal subflow set for each application. Experimental results based on Linux testbed and NS3 validate the performance of EE-MPTCP.

The rest of this paper is organized as follows. Section 2 gives an introduction of MPTCP and presents related works. Section 3 investigates the performance problems of existing approaches and our design motivations. Then the proposed EE-MPTCP algorithm is introduced in Section 4. Section 5 depicts the extensive experimental results as well as their analysis. Finally, Section 6 concludes the paper.

2. Related Works

In this section, we will first introduce the application of MPTCP in IoT. Then, the related works that aims to reduce the energy comsumption are depicted.

2.1. MPTCP in IoT

Currently, some researchers apply MPTCP to the IoT for efficient transmission. Considering the importance of energy for the battery-constrained devices in IoT, Morawski et al. [6] proposed a green MPTCP framework. Aljubayri et al. [7] adapted the Opportunistic Routing (OR) technique to reduce the number of transmissions, and thus reducing MPTCP delay. Pokhrel et al. [5] investigated distributed transfer learning (TL) for MPTCP in the IIoT scenario. Silva et al. [8] adopted MPTCP for traffic offloading for machine-to-machine (M2M) data traffic in the IoT scenario that transmitted between the wireless devices and remote server. These works mainly focus on transmission quality performance improvement and service quality assurance in IoT, and are lack of attention to energy consumption.

2.2. MPTCP with Energy Efficiency

Nowadays, the researchers optimize the energy consumption of MPTCP transmission from two aspects of path scheduling and congestion control.

On the one hand, there are some algorithms related to congestion control. Ref. [9] designed a traffic shifting policy to save the energy consumption of MPTCP, which could divert traffic to the path with lower energy consumption. Ref. [10] improved energy efficiency in data center networks by minimizing the flow completion time and designed additional substream elimination mechanisms to eliminate those substreams that caused high energy consumption due to link sharing. Peng et al. [11] designed different path rate adjustment schemes for different mobile applications. Phame et al. [12] proposed emReno which increase the CWND inversely proportional to energy consumption. That’s, the congestion window of subflows with higher energy consumption is smaller.

On the other hand, there are some algorithms related to packet scheduling algorithm for MPTCP. eMTCP was proposed in [13], which scheduled network traffic according to the energy consumption of each interface to achieve the purpose of energy saving. eMPTCP [14] selected the path for data transmission based on an operating area which was identified through an MPTCP energy consumption model. Arain et al. [15] proposed an MPTCP scheduling melanism based on stochastic optimization to reduce energy consumption and meet the application requirements by limiting the congestion window size and selecting the best path. In [16], a novel energy-aware multipath content transmission method, MPTCP-QE, was proposed, which was driven based on quality of experience (QoE). MPTCP-QE integrated a fast recovery scheme to avoid unnecessary slow start. Palash et al. [17] define three steps to achieve a continuous balance of mobile devices bandwidth requirements and energy efficiency in a round-robin fashion. ES-MPTCP proposed in [18] considered both path throughput and energy consumption, and determines which substreams are used for data transmission with the goal of achieving optimality.

In summary, existing algorithms mainly reduce energy consumption by limiting the transmitted data amount of the path having high energy consumption. However, it is quite challenging to choose appropriate substreams based on different criteria for different application. For instance, what most existing algorithms do is to select paths according to energy consumption. However, the path with lower energy consumption may show poor performance, causing reduced network performance.

3. Problems and Challenges

This section first describes the problems and challenges of MPTCP in heterogeneous wireless networks. Then through the experiments based on Linux tested, the shortcomings of the existing solutions are analyzed.

3.1. Problems

MPTCP can improve transmission performance by using multiple paths concurrently. But in the network environment which changes rapidly at present, especially in heterogeneous networks with asymmetric paths and random packet loss, it is a quite meaningful and difficult thing to develop an efficient scheduling mechanism for MPTCP. Take the network between the inspection robots and the monitor center as an example which can consist of 4G/LTE links and WiFi links simultaneously. In 4G/LTE networks, the base stations are designed with large buffers, causing long transmission delays and near-zero packet loss probability. Meanwhile, for the WiFi path, it comparatively has higher random packet loss rates and short delays. In this case, the packets with a higher sequence number scheduled on the faster path can arrive at the receiver earlier than packets scheduled on the slower path with the lower sequence number. This causes OFO packets delivery. HOL-Blocking means that the buffer is full and the receive window is zero when OFO packets’number is large enough. In the situation, any packets can’t be send by the MPTCP even the network is available. This problem can be alleviated by pre-allocating data packets out of sequence to make them arrive at the receiving end in sequence.

Considering the scenario where an MPTCP connection has two paths which are not connected. The path1’s RTT is set to 30 ms and path2’s RTT is set to 60 ms. Assume they are in congestion avoidance scenario and the CWND is increased by 1 for every RTT. At beginning, path1 and path2 start to send packets #1–5 and packets #6–10 simultaneously. Path1 can complete two rounds of transmission when the packets transferred over path2 arrive at the receiver as shown in Figure 1a. In this case, packets with number of 1–5 are OFO packets. They arrive at the receiver end 20 s ahead of the packets with the number of 6–10.

This problem can be solved by pre-allocating mechanisms. If we pre-allocate packets from 1 to 11 to be in flight on path1, and packets from 12 to 16 to be in flight on path2, all packets from 1–16 can arrive in order ideally as depicted in Figure 1b.

However, designing such a pre-allocating scheduling algorithm is challenging as the network is dynamic, and this well-defined scheme cannot work when there are random packet loss exists. Suppose packet 1 is lost in this condition, it should be retransmitted in the next round as shown in Figure 1c. Then packet disorder problem cannot be avoided in this situation.

Thus, how to achieve efficient packet scheduling with MPTCP to improve energy efficiency, reliability and latency is quite worthy of investigation especially in the context of IoT applications where the paths are heterogeneous and random packets loss exits.

3.2. Problems Validation with Linux-Based Testbed

To validate the problems of the existing algorithms, EE-MPTCP adopts eMTCP [13] and conducts experiments based on Linux testbed which consist of a 3G path and a WLAN path. For the Linux experimental platform, the operating system is Ubuntu 14.04 and the kernel version is 0.90 [19]. The energy consumption is obtained by reading its value from Intel’s CPU RAPL driver at the client side. The network topology is shown in Figure 2 with only two paths, i.e., 3G and WLAN. The WLAN and 3G link characteristics are set with WANem following those in [20]. The bandwidth and RTT of WLAN and 3G are set to [25 Mbps, 25 ms] and [5 Mbps, 65 ms]. The loss rate in WLAN is 1%.

The experimental results are described in Figure 3. Several indicators, namely, the energy efficiency, the throughput and the total energy are considered. We can obtain from the figure that eMTCP consumes more energy than the default MPTCP scheduler when the file size is 800 KB and 900 KB (shown in Figure 4. The reason lies in that eMTCP takes a longer time to complete the transfer with the value of 4.98 s and 3.91 s, respectively, when the flow size is 800 KB. To find the root cause, we further use eMTCP and the default algorithm to track the proportion of packets and throughput transmitted by the WLAN path, and find that the WLAN path of eMTCP and the default scheduler is responsible for about 60.28% and 58.70% of the total data transmission. The reason is that when the low-energy path, i.e., the WLAN path, is idle, the eMTCP algorithm is more inclined to choose the low-energy path serves as data transfer. However, the WLAN path has a higher packet loss rate compared to the 3G path, the obtained throughput of eMTCP over the WLAN path is smaller (1.36 Mbps with eMTCP and 1.75 Mbps with the default scheduler). In addition, timeout retransmissions or fast retransmissions can be caused by packet loss, which will in turn increase overall energy consumption and the flow completion time.

Thus, we can obtain that existing proposals may cause MPTCP performance degradation in some network scenarios and a new path management programme of MPTCP should be proposed. Numerous network status, for example delay, packet loss rate and throughput should be included in the proposed scheme. Network dynamics needs to be followed and the consumption of energy is must be brought down to ensure network data throughput. Therefore, this paper focuses on designing a green MPTCP scheduling algorithm to pander to the challenges of energy and resource constraints of the IoT environment and meet the efficient and reliable data transmission requirements.

4. Our Proposed EE-MPTCP

EE-MPTCP algorithm aims to find the optimal subflow set to transmit data that can maximize the network throughput and minimize energy consumption. In the following, This section is divided into four parts to describe the proposed EE-MPTCP algorithm. First, the target optimization function is present. Then, the energy cost model which calculates the energy consumption of each subflow and the MPTCP data transmission model which is designed to estimate data transmission volume in a certain period of time are introduced. Finally, the workflow of the proposed EE-MPTCP algorithm is described.

4.1. The Target Optimization Function

The obtained throughput and energy consumption of different subflows can be different as the paths of different subflows are heterogeneous. Which subflows are selected for data transmission is a problem we have to solve. Thus, we design a target optimization function as expressed in Equation (1).

$$O_k=\sum _{i=1}^k{\overline{X}}_i-T\ast \sum _{i=1}^k(b_i\ast {\overline{X}}_i+{\theta }_i)(1\le k\le n).$$

(1)

T denotes the transmission duration. ${\overline{X}}_i$ is the average throughput of the $i^{th}$ subflow. ${\displaystyle \sum _{i=1}^k}{\overline{X}}_i$ and ${\displaystyle \sum _{i=1}^k}(b_i\ast {\overline{X}}_i+{\theta }_i)$ represent the sum of average throughput and the sum of the average energy consumption of all subflows, respectively, which will be introduced in the next subsection.

According to Equation (1), we can find that $O_k$ increases with the increase of total average throughput and the decrease of the total energy cost. We need to find $k^{\ast }$ that maximizes $O_k$, i.e., $k^{\ast }=argmax\left\{O_k\right\}$ to guarantee network throughput and the reduction of energy consumption.

4.2. The Energy-Efficiency Model

The amount of data in bits transmitted per joule of energy is called the energy efficiency, which can be calculated by dividing the data transmission volume by the total energy consumption in this transmission. The data volume is calculated by multiplying the average transmission rate $x_r$ and transmission time T. The former $x_r$ is calculated as $\frac{CWN{D}_r}{RT{T}_r}$, where $CWN{D}_r$ and $RT{T}_r$ represent the current congestion window and RTT on path r, respectively. The product of power consumption $P_r\left(x_r\right)$ and duration of connection T is the energy consumption.

Therefore, we can obtain the energy efficiency as:

$$ee=\frac{x_r\ast T}{P_r\left(x_r\right)\ast T}.$$

(2)

where $P_r\left(x_r\right)$ denotes the power consumption on subflow r. Its relationship with throughput $x_r$ is shown in Equation (3), which is widely applied 4G, 3G and WiFi [21].

$$P_r\left(x_r\right)=b_r{x}_r+{\theta }_r{1}_{\{x_r>0\}},$$

(3)

where $b_r>0$. It is the value of the power consumed per unit throughput when receiving data. ${\theta }_r>0$ denotes the instantaneous energy consumption when the interface is activated.

According to Equations (2) and (3), we can obtain that the ratio of data transmission rate to the power consumption is the energy efficiency, that is:

$$ee=\frac{x_r}{{\theta }_r+b_r{x}_r}.$$

(4)

4.3. Modelling of Data Transmission

In MPTCP, each subflow performs a similar congestion control (CA) mechanism as regular TCP. In order to guarantee network fairness and improve throughput, the coupled CA is utilized in MPTCP. Up to now, researchers have proposed many CA algorithms for MPTCP, such as Linked Increases Algorithm (LIA) [22], Weighted Vegas (wVegas) [23], Opportunistic LIA (OLIA) [24] and Balanced LIA (Balia) [25]. The default congestion control algorithm in Linux kernel is OLIA. On each new ACK, OLIA increases the congestion window $w_r$ on subflow r as Equation (5).

$$\frac{w_r/RT{T}_r^2}{{({\sum }_r{\widehat{w}}_r/RT{T}_r)}^2}+\alpha /w_r.$$

(5)

Otherwise, the window $w_r$ is decreased as $w_r/2$ when there are packet losses on subflow r.

$RT{T}_r$ and $w_r$ are RTT and the current CWND on path r of subflow r, respectively. ${\widehat{w}}_r$ is the value of CWND at the equilibrium point. $\alpha$ is a function of $w_r$ and $RT{T}_r$ for all $r\in \Re$, and the ℜ is the set of all subflows. Therefore, OLIA fulfills the goals of MPTCP and provides a better congestion balance than the coupled algorithm [26].

To estimate data amount, we adopt our proposed model present in [4,27], which considers all the four phases during MPTCP’s congestion control procedure, and is modeled in terms of round [28]. The parameters in the transfer model are addressed in

Table 1. List of transfer model’s parameters.

Parameters	Definition
$r_i$	Number of rounds transmitted by subflow r.
$C_r\left(r_i\right)$	The accumulated data segements transmitted until the $r_i^{th}$ round.
$T_r\left(r_i\right)$	The elapsed time until the $r_i^{th}$ round.
$C{W}_r\left(r_i\right)$	Subflow r’s congestion window during the $r_i^{th}$ round.
$SS{T}_r\left(r_i\right)$	Subflow r’s slow start threshold during the $r_i^{th}$ round.
$RT{T}_r$	Subflow r’s RTT.
$P_r\left(r_i\right)$	Subflow r’s packet loss rate in the $r_i^{th}$ round.

.

The average throughput ${\overline{X}}_r$ from the first round until the $r_i^{th}$ round of subflow r is defined as Equation (6).

$${\overline{X}}_r=\frac{C_r\left(r_i\right)}{T_r\left(r_i\right)}.$$

(6)

The $C_r\left(r_i\right)$ can be calculated as the sum of $C_r(r_i-1)$ and the transferred data segments in the current round $C{W}_r\left(r_i\right)-{\displaystyle \sum _{l=0}^{C{W}_r\left(r_i\right)}}P(C{W}_r\left(r_i\right),l)\ast l$, where ${\displaystyle \sum _{l=0}^{C{W}_r\left(r_i\right)}}P(C{W}_r\left(r_i\right),l)\ast l$ represents the number of lost data segments. The process is shown in Equation (7).

$$C_r\left(r_i\right)=C_r(r_i-1)+C{W}_r\left(r_i\right)-\sum _{l=0}^{C{W}_r\left(r_i\right)}P(C{W}_r\left(r_i\right),l)\ast l.$$

(7)

The binomial distribution of the probability that r packets may be lost when the congestion window size is w can be calculated as Equation (8).

$$P(w,r)=\left(\genfrac{}{}{0pt}{}{r}{w}\right)\ast P^r\ast {(1-P)}^{w-r}.$$

(8)

In Equation (6), the time duration $T_r\left(r_i\right)$ is the product of RTT and the transmission rounds as shown in Equation (9).

$$T_r\left(r_i\right)=RT{T}_r\ast r_i.$$

(9)

The $C{W}_r\left(r_i\right)$ is analyzed according to the MPTCP transmission process over each subflow as depicted in Equation (7). More specifically, given the state of the ${(r_i-1)}^{th}$ round, namely, the congestion window $C{W}_r(r_i-1)$, the slow start threshold $SS{T}_r(r_i-1)$ and the probability $P_r(r_i-1)$, we can predict the transmission state of the $r_i^{th}$ round. The detailed process is as follows.

First, we calculate the transition probability in different situations, i.e., all data segments are successfully transmitted, a timeout occurs, the packet is lost and the TCP sender receives three identical acknowledgments.

Let $P_r^0\left(r_i\right)$ define the possibility for the first scenario. $P_r^0\left(r_i\right)$ can be expressed as Equation (10).

$$P_r^0\left(r_i\right)={(1-P)}^{C{W}_r(r_i-1)}.$$

(10)

When there are no more packets to send, the time out will be triggered. This possibility represented as $P_r^1\left(r_i\right)$ can be calculated as Equation (11).

$$P_r^1\left(r_i\right)=\sum _{l=C{W}_r(r_i-1)-3}^{C{W}_r(r_i-1)}P(C{W}_r(r_i-1),l).$$

(11)

When the TCP sender receives three identical acknowledgments, the sender will trigger fast retransmission. This transition possibility $P_r^2\left(r_i\right)$ can be depicted as Equation (12).

$$P_r^2\left(r_i\right)=\sum _{l=1}^{C{W}_r(r_i-1)-4}P(C{W}_r(r_i-1),l).$$

(12)

Then, we calculate the congestion window and SSThresh changes in the corresponding three different situations as described above.

In the case where all data transmission is successful, the congestion window $C{W}_r^0\left(r_i\right)$ can be expressed as Equation (13). When a timeout occurs, the congestion window $C{W}_r^1\left(r_i\right)$ can be shown as Equation (14). If a packet loss occurs, the congestion window $C{W}_r^2\left(r_i\right)$ can be described as Equation (15).

$$C{W}_r^0\left(r_i\right)=\left\{\begin{array}{c}C{W}_r(r_i-1)+1\hfill \\ ifC{W}_r(r_i-1)\ge {\mathrm{SST}}_{\mathrm{r}}({\mathrm{r}}_{\mathrm{i}}-1),\hfill \\ 2\ast C{W}_r(r_i-1)\hfill \\ ifC{W}_r(r_i-1)<{\mathrm{SST}}_{\mathrm{r}}({\mathrm{r}}_{\mathrm{i}}-1).\hfill \end{array}\right.$$

(13)

$$C{W}_r^1\left(r_i\right)=C{W}_r\left(0\right).$$

(14)

$$C{W}_r^2\left(r_i\right)=\frac{C{W}_r(r_i-1)}{2}.$$

(15)

$SS{T}_r^0\left(r_i\right)$ keeps unchanged as shown in Equation (16) when all data transmission is successful. When the time out occurs or packet loss occurs, $SS{T}_r^1\left(r_i\right)$ becomes half of the congestion window as represented in Equation (17).

$$SS{T}_r^0\left(r_i\right)=SS{T}_r(r_i-1).$$

(16)

$$SS{T}_r^1\left(r_i\right)=\frac{C{W}_r(r_i-1)}{2}.$$

(17)

Finally, we count the total amount of data successfully transmitted in these three cases. In each condition, the number of successful data transmissions can be calculated as the product of the congestion window size in this condition and the corresponding occurrence possibility. Thus, the total data amount $C_r\left(r_i\right)$ that was transmitted successfully in the $r_i^{th}$ round can be calculated by the sum of these products. That is

$$C_r\left(r_i\right)=\sum _{m=0}^2\ast P_r^m\left(r_i\right)\ast C{W}_r^m\left(r_i\right).$$

(18)

4.4. The Proposed EE-MPTCP Algorithm

In order to get the optimal subflow set, we propose EE-MPTCP to determine the optimal subflow set k to maximize. To achieve this aim, we first sort the subflows, then select first k subflows from the whole subflows where k ranges from 1 to n, and calculate $O_k$ for each k based on the estimation of the average throughput as well the energy consumption of each subflow. Finally, we select the optimal subflow set that can maximize the target optimization function for data transmission.

(1)

Subflows Sorting

As analyzed above, the higher the energy efficiency shown in Equation (4) of the subflow, the better the performance of the subflow. According to Equation (4), we can obtain that

$$b_r^{{}^{\prime }}=b_r+\frac{{\theta }_r}{x_r}.$$

(19)

According to Equation (19), when one subflow has a higher throughput, its $b_r^{{}^{\prime }}$ is smaller, the energy efficiency of the subflow is higher, the subflow’s performance is better. Thus, we first sort all the available subflows $p_1,p_2,\dots ,p_k,\dots ,p_n$ in the ascending order according to Equation (19), and the sorted collection is $P^{{}^{\prime }}=\{p_1^{{}^{\prime }},p_2^{{}^{\prime }},\dots ,p_k^{{}^{\prime }},\dots ,p_n^{{}^{\prime }}\}$. Then EE-MPTCP adopts the first k subflows in this subflow set for the flow with the file size is $N_t$. We calculate $O_k$ for each $k\in [1,n]$.

(2)

Throughput Calculation

According to the target optimization function depicted in Section 4.1, computing $O_k$ requires first obtaining the average throughput and completion time.

To get the two indicators, for a certain k, we sort the k subflows again according to ascending order of RTT. The sorted set is $S^{{}^{\prime }}=\{s_1^{{}^{\prime }},s_2^{{}^{\prime }},\dots ,s_i^{{}^{\prime }},\dots ,s_k^{{}^{\prime }}\}$. We record the number of rounds of the slowest subflow transmission as $r_0$. $RT{T}_k$ is the RTT of subflow $s_k^{{}^{\prime }}$. Then during $RT{T}_k$, the number of rounds $u_i$ that the other subflows $s_i^{{}^{\prime }}$ can transmit can be calculated as

$$u_i=\lfloor \frac{r_0\ast RT{T}_k}{RT{T}_i}\rfloor .$$

(20)

The amount of data, defined as $C_i\left(u_i\right)$, that path i transmitted from the first round to the $u_i^{th}$ round can be obtained through the multipath transmission model (Lines 8–11 of Algorithm 1). $C_k\left(r_0\right)$ is calculated in the same way as $C_i\left(r_i\right)$ (Lines 5–6 of Algorithm 1). Finally, we can calculate the total data amount as Equation (21).

$$N_{total}=C_k\left(r_0\right)+\sum _{i=0}^{k-1}{C}_i\left(r_i\right).$$

(21)

$$T=r_0\ast RT{T}_k.$$

(22)

Then, we compare the total amount of data $N_{total}$ with the file size $N_t$. When $N_{total}>N_t$, the data amount transmitted over each subflow and each subflow’s number of rounds (Lines 14–15 of Algorithm 1) are recorded. Based on this information, each subflow’s average throughput can be finally obtained (Lines 20–21 of Algorithm 1).

As the transmission time over the slowest subflow decides the total time of data transmission, the completion time T can be produced by $r_0$ and $RT{T}_k$, which is represented in Equation (22). $r_0$ is the number of rounds transmitted by the slowest subflow and $RT{T}_k$ is the RTT of the slowest subflow.

Algorithm 1: Estimate the average throughput based upon loss-throughput model

(3)

The proposed EE-MPTCP algorithm

For a certain k, we calculate each path’s average throughput for all the k subflows and completion time according to section IV-D2. Then $O_k$ can be obtained based on the objective optimization function (Lines 3 in Algorithm 2). Finally, we find the value of $k^{\ast }$ that maximizes $O_k$, i.e., $k^{\ast }=argmax\left\{O_k\right\}$, and the subflows $\{p_1^{{}^{\prime }},p_2^{{}^{\prime }},\dots ,p_{k^{\ast }}^{{}^{\prime }}\}$ are selected for this application. Algorithm 2 shows the process and the set of the selected path is updated every RTT round to make the proposed algorithm can be utilized in the constantly changing network environment.

Algorithm 2: Estimate the optimal subfolw set

5. Performance Evaluation

As the results of the Linux based testbed have been revealed in Section 3.2, we further conduct some experiments to evaluate the performance of the proposed EE-MPTCP by comparing its performance with ROUND-ROBIN, eMPTCP [14] and our earlier proposed DMPTCP [27] based on NS3.13 with more complex network scenarios. The main evaluation index contains the cost of energy, the completion time of flow and energy efficiency.

5.1. Experimental Settings

The experiments are conducted in a network scenario that consists of both LTE and WiFi links whose topology is depicted in Figure 2. In this topology, multiple disjoint subflows function between the inspection robots and the remote monitoring server.

In power grids, the information exchanged between the sensors and the monitor center can be short control messages, pictures and large video files. Thus, we vary the transferred file size from several bytes to 10 MB to simulate different applications in power grids. The traffic pattern in the experiments is consists of short flows, medium size flows and long flows. Specifically, the stream size of short flows is randomly generated and the average value is 57 KB following the pareto distribution. The size of medium-size and long-lived TCP flows are 1 MB and 10 MB, respectively. All are triggered synchronously at 0.1 s.

The parameters of WiFi as well as LTE are set according to the measurements of [29] as shown in

Table 2. System Parameters.

Parameter	Design Range
Data packet size	1460 bytes
TCP ACK size	40 bytes
Number of subflows	8
WiFi capacity	5–10 Mbps
LTE capacity	15–20 Mbps
Data packet size	1460 bytes
WiFi RTT	40–60 ms
LTE RTT	100–200 ms
WiFi loss rate	0–1%
LTE loss rate	0

. The power consumption is calculated according to Equation (3), where $b_r$ and ${\theta }_r$ are set to 137 mW/Mbps and 132.9 mW, 52 mW/Mbps and 1288 mW for WIFI and LTE, respectively. Then, the product of $P_r$ and during T is path r’s energy consumption, and we can add the energy consumption of each interface to get the total energy consumption of the equipment.

5.2. Experimental Results

The eMPTCP preferentially selects WLAN path when it is idle, and only if the energy efficiency of the LTE path meets certain requirements, the LTE path will be taken into consideration. The ROUND-ROBIN uses all available paths in sequential polling. DMPTCP aims to select a collection of paths for faster file transfer. The proposed EE-MPTCP introduces energy consumption based on DMPTCP, and wants to find a set of paths that can minimize the completion time of flow and maximize energy efficiency simultaneously.

The experimental results with short flows are shown in Figure 5. We know that EE-MPTCP’s average completion time is shortest and the total energy consumption is lowest. But, the performance improvement is not remarkable in this network environment because the scale of the short flow is small. All algorithms can finish the transmission in a litter time duration.

Thus, the four algorithms which have concurrent flows with a larger data volume are studied. The results are described in Figure 6 and Figure 7. We can find from these results that as the number and size of concurrent streams increase, the advantages of the proposed EE-MPTCP becomes more and more obvious over existing algorithms. The average completion time of EE-MPTCP is reduced by 79.9% compared to ROUND-ROBIN when there are 30 concurrent long flows. Meanwhile, 79.2% is reduced on the total energy consumption. This benefits from the comprehensive consideration of delay, packet loss, energy consumption and other indicators by EE-MPTCP, making it have stronger adaptability in complex network environment compared with the other three algorithms.

eMPTCP uses only WLAN paths for most of the transmission time, which causes its longest average completion time in most environment. However, its completion time is shorter than ROUND-ROBIN in 30 concurrent long stream scenarios. This is caused by the random packet loss rate on the WLAN link which is not considered for ROUND-ROBIN and eMPTCP. We find that multiple ROUND-ROBIN streams take several times longer than other streams when 30 long streams are transmitted simultaneously, and these flows mostly use LTE paths with relatively high latency, bringing about a longer transmission time than that of normal flows. The reason is that the ROUND-ROBIN doesn’t consider the delay of each path, doesn’t preferentially use paths with a shorter delay, and uses the next flow directly after filling the sending window of one flow. When the WLAN path suffers from packet loss, the data will use the LTE transmission path for a long time.

Besides, we also validate the four algorithms’ performance with varying number of sublows, which consists of one LTE path and multiple WLAN paths. An example traffic pattern is established which have 20 concurrent media flows whose scale is about 1 MB. Figure 8 shows the results. When WLAN paths’number increases, meanwhile, total energy consumption of the four algorithms and the average completion time decreases, and EE-MPTCP performs better than other algorithms in this scenario.

5.3. Industrial Applications

Industry 4.0, also known as the industrial internet of things (IIoT), is the current trend of automation and data exchange in manufacturing [30]. In IIoT, the industrial applications are realized based on Cyber-Physical Production Systems (CPPS), which consists of intelligent entities that can communicate and cooperate with each other. Figure 9 shows an industry network scenario [5] where the devices can communicate and share data with each other. In addition, further analysis is feasible for wireless sensors and intelligent devices to transmit undeveloped data to the remote server.

With the development of IIoT, more and more devices are integrated into it, such as sensing nodes, monitoring devices and control equipments [31] to achieve high improve manufacturing efficiency and product quality. These intelligent entities can generate high volume of traffic, which poses great challenges for networks, like large-scale concurrent access and network congestion [8]. The rapid growth of IIoT traffic also requires a great quantity of energy demand. According to statistics, about 33% of global energy consumption comes from the industrial sector.

Therefore, it is necessary to find a network solution that can tackle the challenges including application QoS and energy management. The proposed EE-MPTCP algorithm can be applied to meet these requirements. As shown in Figure 10, the multi-homed sensors and the remote server run the improved EE-MPTCP. The multiplexing over all available concurrent MPTCP paths can provide aggregated throughput, reduced delay and reliable transmission to the remote server. In addition, as an energy-aware scheduler, EE-MPTCP can save energy and extend the battery life time of the wireless sensors in IIoT.

6. Conclusions and Future Work

An MPTCP energy-efficient optimization scheduling algorithm named EE-MPTCP is proposed in the paper for the IoT-based monitoring system in power grids. EE-MPTCP designed a target optimization function, which is as an index to evaluate the selected path set with EE-MPTCP. The path set that maximizes network throughput and minimizes equipment energy consumption is the optimal one. In order to find the target path set, EE-MPTCP adopts the MPTCP data transmission model and designs a two-tier based scheduling algorithm. The experimental results compared with ROUND-ROBIN, eMPTCP and DMPTCP in a variety of network scenarios prove that EE-MPTCP outperforms these three algorithms. It can improve flow completion time and reduce energy consumption. In addition, the proposed EE-MPTCP can also be used in industry IoT as well as other IoT domains. For the future work, we involve the research on the MPTCP energy consumption optimization algorithms in interactive video streaming and virtual cloud computing environment.