Towards Adaptive Multipath Managing: A Lightweight Path Management Mechanism to Aid Multihomed Mobile Computing Devices

Abstract

With the large scale deployment of multihomed mobile computing devices in today’s Internet, the Multipath TCP (MPTCP) is being considered as a preferred data transmission technology in the future Internet due to its promising features of bandwidth aggregation and multipath transmission. However, MPTCP is more likely to be vulnerable to the transmission quality differences of multiple paths, which cause a “hot-potato” out-of-order arrival of packets at the receiver side, and in the absence of a related approach to fix this issue, serious application level performance degradations will occur. In this paper, we proposes MPTCP-LM${}^3$, a Lightweight path Management Mechanism to aid Multihomed MPTCP based mobile computing devices towards efficient multipath data transmission. The goals of MPTCP-LM${}^3$ are: (i) to offer MPTCP a promising path management mechanism, (ii) to reduce out-of-order data reception and protect against receiver buffer blocking, and (iii) to increase the throughput of mobile computing devices in a multihomed wireless environment. Simulations show that MPTCP-LM${}^3$ outperforms the current MPTCP schemes in terms of performance and quality of service.

1. Introduction

In the past several years, wireless communications systems and networks have experienced a dramatic development [1]. These dramatic advances in wireless communications and networking technologies provide ubiquitous Internet connectivity to a mobile user [2]. Moreover, promoted by the increasing popularity of various wireless networks, mobile devices (i.e., smart phones, laptop computers, and netbooks) are already embedded with multiple Wireless Network Interface Cards (WNICs) and enabled multiple IP features [3,4]. With the multiple IP features (also known as multihoming), the devices can connect to the Internet via multiple IP addresses to provide enhanced and reliable Internet connectivity; they also can simultaneously use several IP addresses to spread data across several end-to-end IP paths. For example, the Apple iOS based products (i.e., iPhone and iPad) [5] and Samsung Galaxy mobile phones (i.e., S9 and S9 Plus) [6] can use both the Wi-Fi and cellular network links to download large files. Such multihomed devices can speed up the transmission data rate and increase their throughput performance by making use of multiple wireless network links simultaneously, enabled by the emerging Multipath TCP (MPTCP) technology [7].

The MPTCP is an extension to traditional single path TCP that enables a multihomed device to allocate data traffic across multiple independent end-to-end paths concurrently through multiple network interfaces [8]. Figure 1 presents a typical MPTCP based multipath communication scenario that involves two multihomed Mobile Devices (MD3 and MD4), which communicate with each other by using both the Wi-Fi and 4G mobile links simultaneously. Such a multipathing feature would be beneficial to the mobile devices to aggregate bandwidth, increase throughput, and enhance communication robustness [9]. Furthermore, MPTCP inherits the standard socket Application Programming Interfaces (APIs) from TCP, which means an MPTCP based device can initiate and establish an MPTCP connection with other devices without making any modifications to today’s Internet applications [10]. With its multipath transmission and backwards compatibility features, the MPTCP is being considered as a preferred transmission technology for data delivery over the current and future Internet [11].

1.1. Motivation

Although applying MPTCP to the multihomed mobile computing devices towards multipath data transmission provides numerous potential benefits for data delivery [12,13,14,15,16,17,18,19], the MPTCP path management mechanism is very simple in RFC6824 [7], and this, in turn, leads to MPTCP being vulnerable to the path quality differences of the multiple paths [20]. With the regular path management mechanism, the MPTCP data traffic can be split and assigned to all the available end-to-end paths for concurrent multipath transmission. Such a full-MPTCP mode may be a useful way of providing a “best-effort” bandwidth aggregation service, but this, in turn, will determine data to be received out-of-order because asymmetric paths in an MPTCP connection are commonly encountered with different transmission characteristics. The out-of-order data arrival phenomenon can be especially problematic when applying MPTCP to a multihomed mobile computing device because today’s mobile computing devices commonly have a poor receiver buffer, due to their very limited memory space. When a large number of out-of-order packets are maintained in the overloaded receiver buffer for reordering, buffer blocking will occur, and thereby, the MPTCP’s performance will encounter a serious degradation (as we will discuss in more detail in the next section).

1.2. Related Works

In recent years, the growing interest in the MPTCP technology has resulted in thousands of peer reviewed publications. In this paper, we categorize the most relevant literature into two groups: MPTCP out-of-order packet arrival cases and MPTCP path management cases.

MPTCP out-of-order packet arrival cases: Xu et al. [9] presented a novel pipeline network coding based MPTCP solution to solve the out-of-order packet arrival problem and thus reduce the packet reordering overhead in multipath transmission. Yang et al. [12] introduced a new scheduling algorithm for MPTCP that mitigates jitter by transmitting packets out-of-order on different MPTCP sub-flows such that they can arrive in-order at the receiver. Alheid et al. [13] compared some different MPTCP congestion control mechanisms used in conjunction with the current TCP packet reordering solutions. Their results demonstrated that the performance of MPTCP could be influenced by the delay difference of multipath paths. Their study also identified combinations of congestion control and packet reordering solutions that give better aggregate goodput performance for different path delay differences. Ou et al. [16] designed a novel out-of-order transmission enabled congestion control and packet scheduling strategy for MPTCP. Xue et al. [17] presented a novel forward prediction based dynamic packet scheduling algorithm for MPTCP, by utilizing maximum likelihood estimation in TCP modeling and taking the packet loss rate and time offset into consideration. Le et al. [18] developed a new forward-delay based packet scheduling algorithm to solve the out-of-order received packet problem in the MPTCP based multipath transmission.

MPTCP path management cases: Wang et al. [20] investigated the performance of MPTCP in real-world Internet scenarios based on the NorNet Testbed. Their study revealed that path management and congestion control settings can present a significant impact on MPTCP’s performance. Kim et al. [21] designed a novel receive buffer based path management mechanism, which managed multiple paths by using both the available receive buffer size and dissimilar characteristics of multiple paths. This path management mechanism was devoted to possibly reducing the out-of-order packet arrival problem and alleviating receive buffer blocking in MPTCP based multipath transmission. Oh et al. [22] proposed a feedback based path failure detection and buffer blocking protection approach for MPTCP. This approach aimed to (i) prevent the usage of underperforming sub-flows in the MPTCP, by using a path failure detection method, and (ii) prevent goodput degradation due to delay differences between paths, by detecting buffer blocking and closing underperforming sub-flows. Our previous work, MPTCP-La/E${}^2$ [4], presented an LDDoS (Low-rate Distributed Denial-of-Service) attack aware energy saving oriented MPTCP extension aiming at optimizing the energy usage while still maintaining user’s perceived quality of cloud multipathing services, by selecting and managing a subset of suitable paths for multipathing. Our previous work (PU)${}^2$M${}^2$ [14] introduced a “Potentially Underperforming” (PU) concept to MPTCP and further designed a novel PU aware multipath management mechanism for MPTCP. The main goals of (PU)${}^2$M${}^2$ were: (i) to prevent the use of potentially failed paths in multipath transmission and (ii) to manage multiple paths in MPTCP adaptively.

1.3. Contributions

In this paper, we propose MPTCP-LM${}^3$, a Lightweight, but highly promising Multipath Management Mechanism for MPTCP. The goals of MPTCP-LM${}^3$ are (i) to possibly mitigate the out-of-sequence packet reception and reduce the delay of packet reordering in multipath transmission, (ii) to aggregate the bandwidth of multiple network paths adaptively and enhance the performance of MPTCP, and (iii) to preserve backward compatibility with the current Internet applications, as MPTCP does. We present simulation results that show that MPTCP-LM${}^3$ can outperform the baseline MPTCP scheme in terms of data sending and receiving times, out-of-order data arrival, and throughput performance. The proposed MPTCP-LM${}^3$ scheme makes fundamental contributions against the state-of-the-art in the literature, in the following aspects:

(i)

It reveals the fact that although each transmission path in MPTCP independently performs data delivery, a poorly performing path (with large delay) can lower the performance of MPTCP, by causing out-of-order data arrival and even receiver buffer blocking in multipath transmission.

(ii)

It introduces a lightweight multipath management mechanism for MPTCP, which allows a path to be removed and reconnected in multipath transmission adaptively, in order to reduce out-of-order data reception and protect against receiver buffer blocking in MPTCP.

Especially, it is worth pointing out that both our previous work (PU)${}^2$M${}^2$ [14] and the MPTCP-LM${}^3$ solution proposed in this paper were devoted to helping MPTCP adaptively manage multiple paths, by removing and reconnecting a transmission path accordingly. However, the (PU)${}^2$M${}^2$ solution was mainly devoted to detecting path failure problems in MPTCP, while the MPTCP-LM${}^3$ solution is mainly devoted to identifying which path would cause the performance degradations in multipath transmission. In addition, (PU)${}^2$M${}^2$ prevented the usage of a PU path according to the path’s delay variations, while in MPTCP-LM${}^3$, a path can be removed/reconnected according to not only the path’s delay performance, but also the path’s congestion window size. This help MPTCP-LM${}^3$ address the limitation of the (PU)${}^2$M${}^2$ solution, avoiding performing worse than the baseline MPTCP if a path has a high level of transmission capacity, but its delay performance becomes worse accidentally or suddenly.

2. Problem Statement

The asymmetric paths in a practical heterogeneous wireless environment with different transmission characteristics are more common and very sensitive to the variations of the real-time wireless condition. Because of this, the out-of-order data arrival will be an inevitable phenomenon in multipath transmission. What is worse, large numbers of out-of-order packets buffered in the constrained receiver buffer will result in the “hot-potato” receiver buffer blocking problem and thereby cause serious throughput performance degradations [22].

Several real-world measurement based studies on MPTCP reported that a hug path quality difference is a very important cause for MPTCP’s low performance. Li et al. [23] performed a comprehensive large scale measurement on MPTCP in high speed mobility scenarios. Their study demonstrated that when Round-Trip Times (RTTs) in separate MPTCP paths differed, a significant out-of-order problem would arise, and the efficiency of MPTCP would be far from satisfactory. Chen et al. [24] presented a measurement based study of MPTCP performance with real wireless settings and background traffic. Their study revealed that the MPTCP performance was affected by path characteristic differences in terms of RTTs and loss rates. In this paper, we are especially interested in understanding how the performance of MPTCP is affected when path characteristics are diverse and caused by path failure. Thus, we present a group of reasonable simulations to answer this question. We also seek to answer why an effective multipath management mechanism is necessary for MPTCP.

To achieve the above purposes, we implemented a sample dual dumbbell network topology in Network Simulator 2 Version 2.35 (NS-2) [25], in which the MPTCP module [26] was included. The network topology is shown in Figure 2. An MPTCP sender and an MPTCP receiver were connected to each other through two independent end-to-end paths (A and B). The propagation delay of the edge links between the sender (or the receiver) and the routers was set to one microsecond with 100 Mbps of bandwidth. The bottleneck link between $R_{1,0}$ and $R_{1,1}$ ($R_{2,0}$ and $R_{2,1}$) was set with a 10 Mbps bandwidth and a 50 milliseconds propagation delay. Path B failed after 30 s of simulation time (we simulated Path B failure by bringing down the bottleneck link between routers $R_{2,0}$ and $R_{2,1}$). The total simulation time was 120 s with infinite FTP flows.

Table 1. The parameters used in the simulations.

Parameter	Value
Application traffic type	FTP traffic
Number of paths	2
Access link bandwidth	100 Mb
Access link delay	1 microsecond
Bottleneck link bandwidth	10 Mb
Bottleneck link delay	50 ms
The link queue type	Droptail
The link queue limit	500 packets
Window size on each path	100
Packet loss on each path	1%

shows the parameters used in the simulations. Moreover, we performed two test cases named Case 1 and Case 2, which were examined as follows to investigate the effect of path transmission stability differences on the performance of MPTCP.

Case 1. The sender turns on its first interface and turns off its second interface at the beginning of simulation, which means always only Path A is used in MPTCP for data transmission.

Case 2. The sender turns on both of its interfaces, which means both Paths A and B are used in MPTCP. However, Path B suffered from network failure after 30 s of simulation time.

Figure 3, Figure 4 and Figure 5 illustrate the performance comparisons in terms of throughput, congestion window (cwnd), and out-of-order Data Sequence Number (DSN) when Case 1 and Case 2 were executed, respectively. As Figure 3 shows, compared with the MPTCP’s throughput in Case 1, the MPTCP’s throughput in Case 2 was gradually lowered from the former (before 30 s of simulation time) to backward (after 30 s of simulation time) due to failure on Path B. When a failure occurred on Path B from the 30th s, it caused back-to-back timeouts on Path B (see Figure 4; the cwnd size on Path B was always set to one after 30 s). As in the failure case, the MPTCP sender tried to transmit data after each of these timeouts on Path B (namely, unnecessary retransmission via the failure path [27]). The resulting receiver buffer blocking in MPTCP, with a great number of out-of-order data (see Figure 5) held in the constrained receiver buffer, prevented the efficient use of Path A for data transmission and resulted in serious throughput performance degradations.

From the above experimental results, we can observe that the MPTCP could achieve lower application level performance even than using a single path when it simultaneously exploited multiple paths that had huge path quality differences. When a path experienced network failure or multiple paths had huge quality differences, the receiver buffer blocking may become an inevitable phenomenon in MPTCP. Unfortunately, the standard MPTCP path management mechanism was very simple; it used all the available paths in the MPTCP connection for concurrent multipath data transmission, without considering that wireless network latency and link failure tended to occur frequently due to mobility, signal fading, and wireless interference. To remedy this situation, this paper proposes a lightweight multipath management mechanism to help MPTCP know not only when, but also how to prevent the use of an underperforming or failure path in multipath transmission. This approach could possibly alleviate the receiver buffer blocking and optimize the throughput performance of MPTCP.

3. System Detail Design

Figure 6 illustrates the architecture of our MPTCP-LM${}^3$ system, which involves an MPTCP sender, an MPTCP receiver, and n multiple paths. As an extension of the standard MPTCP, the MPTCP-LM${}^3$ receiver was responsible for the path collection and reassembling if any MPTCP segment data arrived in the receiver buffer. At the MPTCP-LM${}^3$ sender side, there was an additional MPTCP-LM${}^3$ module compared with the standard MPTCP, which was a Lightweight Multipath Management Mechanism (LM${}^3$) that aimed to (i) prevent the use of an underperforming or failure path in multipath transmission, (ii) collect appropriate paths into MPTCP for smart bandwidth aggregation and parallel transmission, and (iii) reduce the receiver buffer blocking and optimize the quality of service when deploying MPTCP on the multihomed mobile computing devices. Following is the detail of the design of the proposed MPTCP-LM${}^3$ solution.

Simply, in MPTCP-LM${}^3$, the receiver buffer maintaining the out-of-order data for reordering is viewed as a router queuing packets for forwarding. Further, a large number of out-of-order data held in the limited receiver buffer and resulting in receiver buffer blocking can be viewed as a large number of packets held in the overloaded router and resulting congestion. In view of this, MPTCP-LM${}^3$ thinks of buffer blocking occurring at the receiver side as “congestion” occurring in the receiver buffer, like network congestion caused by an overloaded router (the router buffer gets full). Therefore, we borrowed ideas from the classic TCP’s congestion Avoidance (CA) mechanism [28] to design the LM${}^3$ mode for the receiver buffer blocking avoidance. In order to make the paper self-contained, we introduce the general ideas of the CA mechanism as follows.

The CA mechanism is maintained by a TCP sender to adjust the cwnd size and alleviate network congestion. The main operations of the TCP’s CA mechanism running in the congestion avoidance state can be summarized as follows [28]: (i) for every packet acknowledged, the cwnd is additively increased by one Maximum Segment Size (MSS) each RTT; (ii) for any packet loss (three duplicate ACKs are received), set the cwnd value to half the current cwnd size; (iii) on timeout, the cwnd size is reset to one MSS. More simply, the TCP’s CA operations can be expressed by using the following mathematical model:

$$cwn{d}_{new}=\left\{\begin{array}{c}\frac{1}{2}\times cwn{d}_{old},\mathrm{if}\mathrm{a}\mathrm{packet}\mathrm{loss}\mathrm{event}\mathrm{is}\mathrm{detected};\hfill \\ 1\mathrm{MSS},\mathrm{if}\mathrm{a}\mathrm{timeout}\mathrm{event}\mathrm{is}\mathrm{detected},\hfill \end{array}\right.$$

(1)

Inspired by the idea of the above TCP’s CA mechanism, MPTCP-LM${}^3$ was designed to adjust the number of transmission paths intelligently, like the cwnd adjustment in TCP’s CA mechanism, to protect possibly against receiver buffer blocking. To this end, in the connection establishment phase, MPTCP-LM${}^3$ set a buffer blocking counter ($buf\_bloc\_count$) and triggered these counters with receiver buffer blocking events. The $buf\_bloc\_count$ was set with a default value of zero and was incremented by one when a receiver buffer blocking occurred in the receiver buffer. After three consecutive receiver buffer blocking events, the $buf\_bloc\_count$ was reset back to its default value. According to the $buf\_bloc\_count$ value, MPTCP-LM${}^3$ could adaptively abandon or exploit multiple paths, similar to the cwnd adjustment in TCP’s CA mechanism, following the principles below:

(i)

For a case in which $buf\_bloc\_count=1$ (or $buf\_bloc\_count=2$), namely there is a single buffer blocking event occurring in the receiver buffer, MPTCP-LM${}^3$ views this case as “a packet loss event in the TCP congestion avoidance phase”. In this case, the number of paths within the MPTCP connection (denoted as $path\_num$) will be halved, and the half of paths that has high performing transmission quality will be used in multipath transmission.

(ii)

For a case in which $buf\_bloc\_count=3$, namely three consecutive buffer blocking events occurred in the receiver buffer, MPTCP-LM${}^3$ views this case as “a timeout event in the TCP congestion avoidance phase”. In this case, only one path that has the highest transmission quality is used in MPTCP ($path\_num$ is set to one). In addition, the value of $buf\_bloc\_count$ is reset back to zero.

As for path transmission quality, in MPTCP, there are two recommended schedulers used to distinguish multiple paths and assign data to these paths for transmission accordingly, which are the lowest-RTT-first scheduler and the largest-cwnd-first scheduler, which differentiate the transmission quality of multiple paths by using each path’s own RTT and cwnd [29], respectively. In MPTCP-LM${}^3$, the transmission quality of multiple paths is differentiated by a function of two transport layer networking variables (namely RTT and cwnd), enabled by the Simple Additive Weighting (SAW) method [30], which is probably the most popular choice for multiple attribute decision making systems due to its simplicity. By using SAW, the transmission quality of each path in MPTCP-LM${}^3$ is determined by the weighted sum of both the RTT and cwnd values. Assume that path $p_i$ is one of multiple paths within the MPTCP-LM${}^3$ connection; the MPTCP-LM${}^3$’s SAW based path quality evaluation method calculates the transmission quality of $p_i$ by using the following equation,

$$\left.\begin{array}{c}{\mathbb{R}}_{p_i}=\sigma \times cwn{d^{\prime }}_{p_i}+\ell \times RT{T^{\prime }}_{p_i};\hfill \\ s.t.\sigma +\ell =1,\hfill \end{array}\right\}$$

(2)

where ${\mathbb{R}}_{p_i}$ represents the transmission quality of $p_i$. In MPTCP-LM${}^3$, the path with the largest $\mathbb{R}$ value has the best transmission quality. Since SAW mostly reaches an optimal performance when the weighting factors of the system variables are fair [31], therefore, both weighting factors $\sigma$ and ℓ used in Equation (2) were set to a fairness value $\frac{1}{2}$ by default. For real-world networking, the two weighting factors could be changed to other suitable values. Moreover, in Equation (2), $cwn{d}_{p_i}^{\prime }$ and $RT{T}_{p_i}^{\prime }$ are the normalized cwnd and RTT values of $p_i$, which can be calculated by using the following Equations (3) and (4), respectively,

$$cwn{d}_{p_i}^{\prime }=\left(cwn{d}_{p_i}-{\mathrm{H}}_{min}\right)\times \frac{1}{{\mathrm{H}}_{max}-{\mathrm{H}}_{min}},$$

(3)

$$RT{T}_{p_i}^{\prime }=\left({\mathsf{\Pi }}_{max}-RT{T}_{p_i}\right)\times \frac{1}{{\mathsf{\Pi }}_{max}-{\mathsf{\Pi }}_{min}},$$

(4)

where ${\mathrm{H}}_{max}$ and ${\mathrm{H}}_{min}$ are the upper bound and the lower bound of the cwnd size measured on $p_i$, respectively. ${\mathsf{\Pi }}_{max}$ and ${\mathsf{\Pi }}_{min}$ are the upper bound and the lower bound of the RTT values measured on $p_i$, respectively.

Let us suppose n possible paths $\left(p_1,p_2,\cdots ,p_n\right)$ within the MPTCP connection, and let $p_{\mathrm{list}}$, $p_{\mathrm{A}\_\mathrm{list}}$, and $p_{\mathrm{R}\_\mathrm{list}}$ be the collection of all the available paths in the MPTCP connection, the collection of paths activated for multipathing, and the collection of paths removed from multipathing, respectively, in which $\left(p_{\mathrm{A}\_\mathrm{list}},p_{\mathrm{R}\_\mathrm{list}}\in p_{\mathrm{list}}\right)\mathrm{and}{p}_{\mathrm{A}\_\mathrm{list}}\cap p_{\mathrm{R}\_\mathrm{list}}=\varnothing$. The main MPTCP-LM${}^3$ operations, each of which is explained in detail, are as follows:

(i)

Estimating each path’s transmission quality ($\mathbb{R}$) periodically (per RTT) by jointly considering its own cwnd and RTT values;

(ii)

Sorting all the paths in $p_{\mathrm{list}}$ in descending order according to their own $\mathbb{R}$ values;

(iii)

If there is no receiver buffer blocking occurring at the receiver side, assigning traffic over all the paths in $p_{\mathrm{list}}$, as the regular MPTCP does;

(iv)

If there is a receiver buffer blocking event detected, but $buf\_bloc\_count<3$, halving the number of paths in multipath transmission, and then incrementing the value of $buf\_bloc\_count$ by one, by using the following Equations (5) and (6),

$$path\_num\leftarrow ⌈\frac{1}{2}\times path\_num⌉.$$

(5)

$$buf\_bloc\_count\leftarrow buf\_bloc\_count+1.$$

(6)

In this case, the top half of paths in the $p_{\mathrm{A}\_\mathrm{list}}$ will be used in multipath transmission (there is $p_{\mathrm{A}\_\mathrm{list}}=p_{\mathrm{list}}$ at the beginning of MPTCP based multipathing).

(v)

If there is a receiver buffer blocking detected and $buf\_bloc\_count=3$, reducing the number of paths to one, then resetting the value of $buf\_bloc\_count$ back to zero, by using the following Equations (7) and (8),

$$path\_num\leftarrow 1.$$

(7)

$$buf\_bloc\_count\leftarrow 0.$$

(8)

In this case, only the first path in $p_{\mathrm{A}\_\mathrm{list}}$ can be used for data transmission.

By using the above operations, MPTCP-LM${}^3$ can intelligently aggregate the bandwidth of multiple paths for data transmission while possibly mitigating the receiver buffer blocking problem. Moreover, in order to maximize the network resource utilization possibly, in MPTCP-LM${}^3$, a path $p_j$ in $p_{\mathrm{R}\_\mathrm{list}}$ can be put into $p_{\mathrm{A}\_\mathrm{list}}$ for multipath transmission, if its current transmission quality, ${\mathbb{R}}_{p_j}$, is not lower than the average transmission quality of other high performing paths (in order to make sure $p_j$ has little transmission quality difference compared to other high performing paths), namely,

$${\mathbb{R}}_{p_j}\ge \frac{1}{\mathrm{count}\left(p_{\mathrm{A}\_\mathrm{list}}\right)}\times \sum _{k=1}^{\mathrm{count}\left(p_{\mathrm{A}\_\mathrm{list}}\right)}{\mathbb{R}}_{p_k},$$

(9)

where $\mathrm{count}\left(p_{\mathrm{A}\_\mathrm{list}}\right)$ is the number of paths in $p_{\mathrm{A}\_\mathrm{list}}$. $p_k$ is the $k^{\mathrm{th}}$ path in $p_{\mathrm{A}\_\mathrm{list}}$. The pseudocode of multipath management mechanism used in MPTCP-LM${}^3$ is presented in Algorithm 1. For convenience, we adopted the following notations used in Algorithm 1 (as shown in

Table 2. Notations used in the MPTCP-LM${}^3$ based multipath management algorithm.

Notation	Description
$p_{\mathrm{list}}$	The collection of all the available paths in the MPTCP connection
$p_{\mathrm{A}\_\mathrm{list}}$	The collection of paths activated for multipathing
$p_{\mathrm{R}\_\mathrm{list}}$	The collection of paths removed from multipathing
$p_{\mathrm{list}(i)}$	The $i^{\mathrm{th}}$ path within $p_{\mathrm{list}}$
$p_{\mathrm{A}\_\mathrm{list}(j)}$	The $j^{\mathrm{th}}$ path within $p_{\mathrm{A}\_\mathrm{list}}$
$p_{\mathrm{R}\_\mathrm{list}(k)}$	The $k^{\mathrm{th}}$ path within $p_{\mathrm{R}\_\mathrm{list}}$

).

For Algorithm 1, the time complexity is primarily influenced by the first “for” statement (from Line 2 to Line 4), the second “for” statement (from Line 16 to Line 21), and the sort algorithm in Line 5. For the two “for” statements, the time complexity of each “for” statement is $\mathrm{O}\left(n\right)$ (we assumed the second “for” statement was executed). As for the sort algorithm, the time complexity was $\mathrm{O}\left(\mathrm{n}{log}_2\mathrm{n}\right)$ because the QuickSort algorithm was used for sorting the paths. Since there was $f\left(\mathrm{n}\right) = 2\mathrm{n}+\mathrm{n}{log}_2\mathrm{n}$, therefore the time complexity of this algorithm was $\mathrm{O}\left(\mathrm{n}{log}_2\mathrm{n}\right)$. More simply, the total number of paths collected by Algorithm 1 can be expressed by using the following mathematical model:

$$path\_num=\left\{\begin{array}{c}path\_num,\mathrm{if}buf\_bloc\_count=0;\hfill \\ ⌈\frac{1}{2}\times path\_num⌉,\mathrm{if}buf\_bloc\_count<3;\hfill \\ 1,\mathrm{if}buf\_bloc\_count=3.\hfill \end{array}\right.$$

(10)

Algorithm 1. MPTCP-LM${}^3$ based multipath management algorithm.
	Initialization:
	$p_{\mathrm{A}\_\mathrm{list}}=p_{\mathrm{list}}$;
	$p_{\mathrm{R}\_\mathrm{list}}=\varnothing$;
	$path\_num=\mathrm{count}\left(p_{\mathrm{list}}\right)$;
	$buf\_bloc\_count=0$.
1:	use all the paths in $p_{\mathrm{A}\_\mathrm{list}}$ for multipathing;
2:	for$\left(i=1,i\le \mathrm{count}\left(p_{\mathrm{A}\_\mathrm{list}}\right),i++\right)$do
3:	calculate the $\mathbb{R}$ value of $p_{\mathrm{A}\_\mathrm{list}(i)}$ by using Equation (2);
4:	end for
5:	sort the paths in $p_{\mathrm{A}\_\mathrm{list}}$ in descending order according to their own $\mathbb{R}$ values;
6:	if there is a receiver buffer blocking detected then
7:	set $buf\_bloc\_count\leftarrow buf\_bloc\_count+1$;
8:	if $buf\_bloc\_count<3$, then
9:	set $path\_num\leftarrow ⌈\frac{1}{2}\times path\_num⌉$;
10:	end if
11:	if $buf\_bloc\_count=3$, then
12:	set $path\_num\leftarrow 1$;
13:	end if
14:	end if
15:	if$\left(\mathrm{count}\left(p_{\mathrm{R}\_\mathrm{list}}\right)>0\right)$, then
16:	for $\left(j=1,j\le \mathrm{count}\left(p_{\mathrm{R}\_\mathrm{list}}\right),j++\right)$ do
17:	calculate the $\mathbb{R}$ value of $p_{\mathrm{R}\_\mathrm{list}(j)}$ by using Equation (2);
18:	if ${\mathbb{R}}_{p_{\mathrm{R}\_\mathrm{list}(j)}}\ge \frac{1}{\mathrm{count}\left(p_{\mathrm{A}\_\mathrm{list}}\right)}\times {\displaystyle \sum _{i=1}^{\mathrm{count}\left(p_{\mathrm{A}\_\mathrm{list}}\right)}}{\mathbb{R}}_{p_{\mathrm{A}\_\mathrm{list}(i)}}$
19:	move $p_{\mathrm{R}\_\mathrm{list}(j)}$ into $p_{\mathrm{A}\_\mathrm{list}}$;
20:	end if
21:	end for
22:	end if
23:	allocate the MPTCP data across the path(s) in $p_{\mathrm{A}\_\mathrm{list}}$.

4. Performance Evaluation

4.1. Simulation Topology

The performance evaluation was carried out on Network Simulator 2 Version 2.35 (NS-2) [25] with the MPTCP module [26]. The experiments considered a typical MPTCP based device-to-device communication scenario presented in Figure 7. As the figure shows, an MPTCP sender and an MPTCP receiver are connecting with each other via three independent transmission paths (denoted A, B, and C). Path A’s bottleneck has a 10 Mbps bandwidth and 10–20 ms propagation delay. Path B’s bottleneck has a 11 Mbps bandwidth and 10–20 ms propagation delay. Path C’s bottleneck has a 2 Mbps bandwidth and 10–60 ms propagation delay. In the simulation, Path C will experience a short term failure from 10 to 30 s and a complete failure after 50 s of simulation time (we simulated Path C failure by bringing down the bottleneck link between routers $R_{3,1}$ and $R_{3,2}$). The parameters shown in

Table 3. Path parameter configuration used in the simulation.

Network Parameters	Path A	Path B	Path C
Wireless access bandwidth	10 Mbps	11 Mbps	2 Mbps
Wireless link delay	10–20 ms	10–20 ms	10–60 ms
Wireless link queue type	Droptail	Droptail	Droptail
Core network bandwidth	100 Mbps	100 Mbps	100 Mbps
Core network delay	30 ms	30 ms	30 ms
Uniform loss rate	1–2%	1–4%	1–6%
Markov loss rate	1%	1%	1%

were used for configuring multiple paths. The other parameters of the MPTCP used the default values specified in NS-2.

In the experiment, all the wireless access links were attached to a Uniform Path-Loss (UPL) model and a Two-state Markov Loss (TML) model in order to simulate the highly frequent and bursty frame loss in the data link layer. The UPL and TML models were used to represent the distributed loss caused by wireless noise interference and the infrequent continuous loss caused by wireless signal fading, respectively [32]. In addition, in order to simulate the complex behaviors of Internet traffic [33], we attached each of the three paths to a VBR (Variable Bit-Rate) traffic generator to send the VBR cross-traffic to its corresponding sink. The packet size of the VBR cross-traffic utilized in the simulation was chosen as follows [34]: 49% were 44 bytes in length, 1.2% 576 bytes, 2.1% 628 bytes, 1.7% 1300 bytes, and the other 46% 1500 bytes in length, in which 90% of the VBR cross-traffic was transmitted by TCP technology and the remaining 10% by UDP technology. The VBR cross-traffic on each of the three paths consumed 0–50% of the access link bandwidth. The simulation time was set to 100 s with infinite FTP traffic.

4.2. Simulation Results

We here compare the performance of our MPTCP-LM${}^3$ with the baseline MPTCP [7] and our previous work (PU)${}^2$M${}^2$ [14]. For convenience, we portray the results of MPTCP-LM${}^3$ as “MPTCP-LM${}^3$” in the result figures, and the results when applying the baseline MPTCP and (PU)${}^2$M${}^2$ schemes are portrayed as “the baseline MPTCP” and “MPTCP + (PU)${}^2$M${}^2$”, respectively.

(1) Sending and receiving data DSN: Figure 8 shows the comparison results of data sending and receiving times when the baseline MPTCP, MPTCP + (PU)${}^2$M${}^2$, and MPTCP-LM${}^3$ were used, respectively. In order to better illustrate the comparison, the results between t = 30 s and t = 100 s are illustrated (including the no path failure phase (between 30 s to 50 s) and path failure phase (after 50 s)), representative for all the simulation results. From the figure, we can observe that between 30 and 50 s of simulation time, MPTCP-LM${}^3$ and MPTCP + (PU)${}^2$M${}^2$ performed at a lower level of data sending and receiving times than the baseline MPTCP. This was because the baseline MPTCP fully utilized the three paths for data transmission (Path C was available and reused in multipath transmission after 30 s). In both MPTCP-LM${}^3$ and MPTCP + (PU)${}^2$M${}^2$, Path C may be prevented from multipathing due to the quality gap (compared with Paths A and B).

However, MPTCP-LM${}^3$ and MPTCP + (PU)${}^2$M${}^2$ could achieve a higher level of data sending and receiving times than the baseline MPTCP after 50 s. This was because in both MPTCP-LM${}^3$ and MPTCP + (PU)${}^2$M${}^2$, the sender only selected the stable paths for multipath transmission, while in the baseline MPTCP, the path failure declaring was inherited from TCP’s operations (in the TCP, the time required to declare a broken path needs at least $1+3+6+12+24=46$ s ($TcpMaxDataRetransmissions=5$ and $RTO=1$ s)). That is, between 50 and 100 s, the broken Path C could also be used in multipath transmission, which caused the interruptions in Paths A and B and, thus, constrained the sender from transmitting data traffic.

Besides, we can also see that between 50 and 70 s of simulation time, MPTCP + LM${}^3$ achieved a lower level of data sending and receiving times than the MPTCP + (PU)${}^2$M${}^2$ scheme. This was because MPTCP + LM${}^3$ detected path failures based on the counts of receiver buffer blocking, while MPTCP + (PU)${}^2$M${}^2$ was based on the path’s delay variations, which means MPTCP + LM${}^3$ could possibly find out and prevent the usage of a broken path or a poorly performing path more slowly than MPTCP + (PU)${}^2$M${}^2$. However, in MPTCP + (PU)${}^2$M${}^2$, a path could be removed/reconnected frequently if its delay performance was getting worse accidentally or suddenly. This limitation made MPTCP + (PU)${}^2$M${}^2$ not able to provide stable transmission efficiency and perform worse than MPTCP + LM${}^3$ in the rest of the simulation time (between 70 and 100 s).

(2) Out-of-order DSN: The Out-Of-Order DSN (OOO DSN) is a good metric to select for studying and analyzing the performance of multipath protocols. The OOO DSN can be attained by the offset between the DSNs of two consecutively received MPTCP data chunks (the difference between the DSN of the current MPTCP chunk and that of the latest received MPTCP chunk). Figure 9 shows a comparison of out-of-order chunks among the baseline MPTCP, MPTCP + (PU)${}^2$M${}^2$, and MPTCP-LM${}^3$. As the figure shows, the baseline MPTCP scheme generated more out-of-order chunks and required increased unnecessary packet reordering than both MPTCP + (PU)${}^2$M${}^2$ and MPTCP-LM${}^3$. This was because the baseline MPTCP’s scheduler assigned data traffic over all the paths, without considering that a poorly performing path or a failure prone path would cause extremely great numbers of out-of-order chunks for reordering.

In contrast, both MPTCP + (PU)${}^2$M${}^2$ and MPTCP-LM${}^3$ could detect a broken/poorly performing path and select a group of stable paths for data transmission. In this way, they could possibly allocate data traffic over the stable and high performing paths and thus reduce the out-of-order data reception. However, it was noted that MPTCP + (PU)${}^2$M${}^2$ often generated more out-of-order data chunks than MPTCP-LM${}^3$. This was because in MPTCP + (PU)${}^2$M${}^2$, the paths selected for multipath transmission would be frequently changed and sensitive to paths’ delay variations, while in MPTCP-LM${}^3$, the path group, which was jointly determined by the paths’ delay and cwnd characteristics, was relatively stable and changed little. When comparing the three schemes, we can see that the peak out-of-order data reception at the receiver was up to $1.40\times {10}^6$ when using the baseline MPTCP, while it was approximately $1.39\times {10}^5$ when using both MPTCP + (PU)${}^2$M${}^2$ and MPTCP-LM${}^3$.

(3) End-to-end delay: Figure 10 shows a comparison of end-to-end delay performance when using the baseline MPTCP, MPTCP + (PU)${}^2$M${}^2$, and MPTCP-LM${}^3$ schemes, respectively. As we analyzed earlier, both MPTCP-LM${}^3$ and MPTCP + (PU)${}^2$M${}^2$ could prevent an underperforming path (or a broken path) from multipathing, and they could thereby prevent unnecessary retransmission in the underperforming or broken paths. This made MPTCP-LM${}^3$ and MPTCP + (PU)${}^2$M${}^2$ reduce the unnecessary retransmission delay. Moreover, both MPTCP-LM${}^3$ and MPTCP + (PU)${}^2$M${}^2$ could possibly ensure that the MPTCP segments arrive at the receiver in the right order. This helped the two schemes reduce the packet reordering delay. As a result, MPTCP-LM${}^3$ and MPTCP + (PU)${}^2$M${}^2$ obtained a lower level of delay than the baseline MPTCP.

However, MPTCP + (PU)${}^2$M${}^2$ performed worse than MPTCP-LM${}^3$ because more out-of-order data reception in MPTCP + (PU)${}^2$M${}^2$ required increased reordering than MPTCP-LM${}^3$. Moreover, in MPTCP + (PU)${}^2$M${}^2$, the paths used in multipath transmission often changed, and as a result, the data traffic could be offloaded from one path to another frequently, which incurred additional processing time overhead. When comparing the three methods in terms of the cumulative average delay (with a total of 100 s of simulation time), the MPTCP-LM${}^3$ scheme’s cumulative average delay was approximately 4.35% lower than that of the baseline MPTCP and 1.03% lower than that of MPTCP + (PU)${}^2$M${}^2$.

(4) Jitter comparison: Jitter can be defined as the amount of packet delay variation in the end-to-end data traffic transmission time. Jitter is a well suited metric to evaluate and analyze the temporal performance of a multipath transmission protocol. A higher level of jitter is more likely to occur on a poorly performing transport technology and vice versa. Figure 11 shows a comparison of the jitter performance when the three schemes were used. Although the path management behaviors of both MPTCP-LM${}^3$ and MPTCP + (PU)${}^2$M${}^2$ (removing or reconnecting a path in multipath transmission) influenced the jitter performance, which was occasionally less than that of the baseline MPTCP, we argue that this was acceptable and worthy of achieving high level performance since MPTCP-LM${}^3$ and MPTCP + (PU)${}^2$M${}^2$ could adaptively select appropriate paths for multipathing. In contrast, the baseline MPTCP, as previously mentioned, was bound to out-of-order data reception (due to path quality characteristics) and unnecessary retransmission (due to path failures), resulting in a lower quality of service than both MPTCP-LM${}^3$ and MPTCP + (PU)${}^2$M${}^2$.

When comparing the MPTCP + (PU)${}^2$M${}^2$ and MPTCP-LM${}^3$ methods, it was noted that MPTCP + (PU)${}^2$M${}^2$ more or less generated a higher level of jitter than MPTCP-LM${}^3$. This was because MPTCP + (PU)${}^2$M${}^2$ frequently deactivated or reactivated a path for data transmission, which incurred the additional overhead for data offloading and reordering and, thus, increased the amount of packet delay variations. In contrast, MPTCP + (PU)${}^2$M${}^2$ provided a more stable path group for data transmission than MPTCP + (PU)${}^2$M${}^2$, and it could possibly reduce the overhead of data offloading and reordering and correspondingly generate a lower level of jitter performance than MPTCP + (PU)${}^2$M${}^2$.

(5) Throughput performance: Figure 12 shows a comparison of throughput performance when using the three schemes. MPTCP + (PU)${}^2$M${}^2$ and MPTCP-LM${}^3$ prevented the usage of underperforming or broken paths in multipath transmission, which could not only reduce the probability of receiver buffer blocking, but also reduce transmission interruptions in the other high performing paths. Therefore, both MPTCP + (PU)${}^2$M${}^2$ and MPTCP-LM${}^3$ performed better than the baseline MPTCP. When comparing MPTCP + (PU)${}^2$M${}^2$ and MPTCP-LM${}^3$, it was noted that the MPTCP-LM${}^3$ achieved higher association average throughput than the MPTCP + (PU)${}^2$M${}^2$. This was because MPTCP + (PU)${}^2$M${}^2$ detected an underperforming or a broken path by monitoring the delay changes of paths. This path estimation method was very sensitive to variations of delay and could cause throughput performance degradation because of frequently changing the paths in multipath transmission. In contrast, MPTCP-LM${}^3$ declared an underperforming or a broken path by monitoring the receiver buffer blocking, and in this way, it helped MPTCP-LM${}^3$ provide stable transmission performance.

In order to illustrate the throughput comparison better, we calculated the cumulative average throughput of the three schemes respectively, by averaging the total average throughput values in a total of 100 s of simulation time. The subfigure in Figure 12 shows a comparison of cumulative average throughput when the three schemes were used, respectively. As the subfigure shows, MPTCP-LM${}^3$ performed the best among the three schemes compared. However, it is noteworthy that the cumulative average throughput of MPTCP-LM${}^3$ was about 2.8 Mbps, which corresponded only to 12% of the aggregated bandwidths of the three access links (10 Mbps + 11 Mbps + 2 Mbps = 23 Mbps). We argue that this phenomenon was caused by several reasons: (i) the VBR competing traffic could consume a certain amount of each path’s access bandwidth; (ii) the two wireless loss models attached to each of the three access links could lead to the degradation of MPTCP throughput performance; (iii) the path failures on Path C will inevitably caused transmission interruption in Paths A and B and, thus, lowered the MPTCP throughput performance; and (iv) the out-of-order data arrival would actually degrade the performance in most cases because of the path quality differences. Still, the MPTCP-LM${}^3$ scheme’s cumulative average throughput was 27.97% higher than that of the baseline MPTCP and 3.68% higher than that of MPTCP + (PU)${}^2$M${}^2$.

5. Discussion

In this paper, we proposed a lightweight path management mechanism to aid multihomed MPTCP based mobile computing devices towards efficient multipath data transmission, inspired by the TCP congestion avoidance mechanism. The proposed solution can be easily merged with the current MPTCP variants to remove and reconnect the connection in MPTCP adaptively. In this section, we highlight the limitations of our solution and give some interesting directions for future work. We encourage the researchers who are interested in this field to pay attention to and discuss the following interesting challenges.

• In practice, the number of connection interfaces on the end-devices may not be too large, which may constrain the success of our proposal in today’s Internet architectures. We argue that end-devices in the future Internet may be equipped with many network interfaces and can see multiple access links. The authors hope to attract more researchers to discuss this topic.
• In the Performance Evaluation Section, we discussed the possible reasons why the throughput performance of all the MPTCP variants was far from satisfactory. We noticed that in a sample dual dumbbell simulation topology (see Section 2), the cumulative average throughput of MPTCP was only close to 1.5 Mbps, which corresponded only to 15% of the bottleneck bandwidth of Path A (10 Mbps). For these negative simulation results, we argue that the NS-2 model of MPTCP may not be able to reflect the MPTCP implementation fully. We encourage more researchers to pay attention to and discuss this controversial problem.

6. Conclusions and Future Work

Mobile computing devices attached to more than one wireless network interface can improve their transmission performance by making use of the MPTCP technology. However, the path management mechanism in MPTCP is very simple and vulnerable to the quality differences of multiple paths in heterogeneous wireless environments. As a remedy, this paper presented a lightweight multipath management mechanism for MPTCP (MPTCP-LM${}^3$) necessitating the following aims: (i) optimizing MPTCP path management mechanism and possibly preventing an underperforming path from multipathing, (ii) reducing the out-of-order data reception and alleviating the receiver buffer blocking problems in MPTCP, (iii) improving the throughput performance and quality of service of multipath transmission. The simulation results demonstrated that MPTCP-LM${}^3$ outperformed the baseline MPTCP and MPTCP + (PU)${}^2$M${}^2$ in terms of data transmission service quality.

By carrying out simulations on top of NS-2, we noted a limitation that the throughput performance of all the three MPTCP variants was far from satisfactory, as we discussed in the Performance Evaluation Section. Recent research argues that there is an open-source implementation of multipath TCP that exists and is well maintained in the Linux kernel [35]. Our future work will develop a Linux kernel based hardware test-bed, then implement the proposed MPTCP-LM${}^3$ solution in the test-bed, and provide real experimental results.

Furthermore, due to the strict layering principles, MPTCP-LM${}^3$ can only use the networking parameters at the transport layer to estimate each path’s transmission quality and thereby make a “best-effort” transmission behavior. Considering the fact that variations of the wireless channel, such as channel fading and co-channel interference, are the extremely influencing factors in the wireless transmission, our future work will be devoted to the optimization of our MPTCP-LM${}^3$ solution by jointly considering the cross-layer design and “smart collaborative networking” method [36,37,38,39]. Then, a comprehensive wireless network environment including many and promising wireless technologies (i.e., Bluetooth, LoRaWAN, SigFox, etc.) will be considered, and the comparison with the state-of-the-art MPTCP schemes will be investigated in the performance evaluations.