A Fine-Grained Network Congestion Detection Based on Flow Watermarking

Abstract

With the rapid development of the network, how to effectively reduce the dynamic delay and improve the performance of the network is an important and challenging problem. Specifically, network congestion is one of the key factors that hurt the network performance, so real-time detection of the network congestion is critical for recovering the network failure quickly. Current research in congestion detection mainly faces the problems of occupying extra bandwidth, decreasing the ratio of the effective payload of the packet, increasing the burden of the switches, etc. In this paper, we apply flow watermarking to network congestion detection and propose a fine-grained network congestion detection method based on flow watermarking. We also combine it with the eBPF (extended Berkeley Packet Filter) to improve the performance of congestion detection. Theoretical analysis and experimental results show that the changes in network status can be reflected in real-time and accurately in the watermark decoding information. The network congestion detection based on flow watermarking can detect network status on a small time scale and realize low-overhead and easily deployed congestion detection.

1. Introduction

As the network architecture and functions become more and more complex, both users and operators require better network performance; however, because the rapid growth of network traffic incurs traffic collision, blindly increasing the network bandwidth cannot effectively improve the network performance. Network management faces new challenges; it requires quick awareness of network status and response to the network failures as soon as possible to decrease the dynamic delay. Therefore, efficient network congestion detection is particularly important.

Network congestion detection uses software or hardware tools to collect the real-time network status and provides data support for network fault location and routing calculation, which is the basis of network management. Researchers have proposed many network congestion detection methods but there are some shortcomings. Active network congestion detection introduces probes, which occupy the extra bandwidth. Passive network congestion detection captures the packets information through detection points, which lacks the view of global network status. In the SDN (Software Defined Network) -based network congestion detection, the controller frequently communicates with switches and installs flow entries, which increases the number of flow entries and may occupy bandwidth, computing and storage resources. The INT (In-band Network Telemetry) -based network congestion detection encapsulates telemetry commands and data into packets, which reduces the ratio of effective packet payload and increases the burden of switches.

The industry has also conducted in-depth research on network congestion detection. Huawei proposed AI Fabric, a datacenter switch based on high-performance AI chips [1]. The switch can collect traffic characteristics and network status information in real-time and achieve zero packet loss. Therefore, AI Fabric can identify faults in seconds timescale and locate faults in minutes timescale; however, the duration of burst traffic in the current datacenter network is usually a few seconds or milliseconds, so it is not effective in dealing with the micro-burst traffic. More fine-grained and low-overhead network congestion detection is required to cope with the changes in network status on a small timescale, and to achieve more efficient network congestion detection.

Network flow watermarking [2] is a technique that determines whether there is a flow association between sender and receiver by changing certain characteristics of network traffic to embed watermark information; it is often used for springboard detection and attack source tracing. Time-based network flow watermarking technique encodes different information by modifying the IPD (Inter-Packet Delay), which embeds management data into the flow transmission process only by delaying sending time of packets.

Network congestion may destroy the original feature of the watermark. Figure 1 shows the distortion of IBF (Interval-Based Fingerprinting) [3], which is a time-based network flow watermarking technique when facing network congestion. Before the distortion, the packets of the second interval are moved into the third interval for transmission. The number of packets in the second interval is less than the average number of packets in the three consecutive intervals, which present data ‘1’. After network congestion occurs, the packets in the first interval are delayed until the second interval, and the packets in the third interval are lost. At this time, the number of packets in the second interval is greater than the average value in the three intervals, which presents data ‘0’. The variational network status changes the original sparsity of packets and destroys the watermark they carried. The watermark decoding information can roughly feed back the changes in network status.

Therefore, we apply the time-based flow watermarking technique to network congestion detection. But the flow watermarking based on user mode cannot cope with high-speed traffic and hurts the network performance. The eBPF is a general execution engine that provides a general ability to execute specific code efficiently and safely based on system or program events, which enables user mode to implement customized functions; thus, it is very suitable for implementing network flow watermarking for high-rate traffic.

As shown in

Table 1. Comparison of network congestion detection methods.

Classification	Carrier	Whether to Occupy Extra Bandwidth	Whether to Modify Packet Content	Detection Speed
Watermark-based detection	IPD	No	No	Fast
Active detection	Probe packets	Yes	No	Slow
Passive detection	Mirrored packets	No	No	Slow
SDN-based detection	Probe packets, etc.	Yes/No	No	Slower
INT-based detection	Packet header field	Yes/No	Yes	Fast

, the following evaluation indicators are selected to compare the existing network congestion detection methods with the network congestion detection based on flow watermarking:

(1)

Carrier: Indicating the detection method.

(2)

Whether to occupy extra bandwidth: Occupying extra bandwidth will cause the ‘observer effect’.

(3)

Whether to modify the packet content: Modifying the packet content will bring security problems and additional switch overhead.

(4)

Detection speed: Reacting to the efficiency of network congestion detection.

In this paper, we propose a fine-grained network congestion detection method-based flow watermarking, which is implemented in the eBPF environment; it improves the accuracy and speed of network congestion detection with low overhead and easy deployment, which provides theoretical and practical support for network management. The main contributions of this paper are as follows:

• First, we design a network congestion detection model based on flow watermarking and theoretically analyze the network congestion detection method based on flow watermarking. The flow watermarking is applied to network congestion detection for the first time, which doesn’t occupy extra bandwidth, change the payload of the packet, and increase the processing burden.
• Second, we propose a fine-grained network congestion detection algorithm to solve the problem that the congestion caused by micro-burst traffic has the characteristics of short duration and rapid change, which makes it difficult to detect by conventional congestion detection methods; it can detect network congestion on a small time scale and we can obtain current network status by analyzing the watermark decoding information.
• Third, we introduce eBPF into network congestion detection based on flow watermarking to enable it easy to employ. The experimental results show that the flow watermarking is sensitive to the changes in network status. The fine-grained network congestion detection algorithm can more accurately judge network status.

2. Related Work

2.1. Network Congestion Detection

Active network congestion detection [4,5] obtains network information by analyzing the behavior of probes actively constructed under the changes in network status, such as ping, traceroute and iPerf. The operation is simple and fast, but there is an ‘observer effect’; it injects probes, and adds extra bandwidth overhead to the network, which may change the original network status.

Passive network congestion detection [6,7] captures the information of the packets through the detection point and sends it to the server to obtain traffic characteristics and network status, such as NetFlow, sFlow and IPFIX; it does not generate extra bandwidth overhead, but the design of mechanisms such as sampling will also sacrifice accuracy of detection.

The SDN-based network congestion detection [8,9,10] can detect network delay, packet loss rate, etc. Adrichem et al. [11] proposed a measurement scheme OPENnetmon, which actively polls traffic and destination switches and monitors link delay and packet loss through the controller. Yu et al. [12] proposed a delay monitoring framework SLAM, which triggers control messages by actively sending probe packets, and analyzes the time when the messages arrive at the controller to obtain the delay. Fu et al. [13] proposed an efficient passive lightweight predictor EPLE, which uses the Openflow protocol for detecting packet loss. The passive detection method reduces the overhead.

The INT-based network congestion detection [14,15,16] exploits the programmability of the network data plane to collect network status information. Zhu et al. proposed a packet-level network telemetry system, Everflow [17], which actively constructs telemetry packets to obtain network status information; it has the advantages of flexibility, efficiency, and configurability. HULA [18] uses special probes to collect global link utilization information and update it periodically, which may lead to a large bandwidth overhead. Pan et al. proposed INT-path [19] by encapsulating the source routing label into the probe packets, which can telemetry the specified network path status.

2.2. Network Flow Watermarking

The time-based network flow watermarking techniques includes the packet timing-based flow watermarking techniques and the packet interval-based flow watermarking techniques. The packet timing-based flow watermarking techniques [20] embed watermark by modifying time-dependent characteristics, which is vulnerable to multi-flow attacks, and network events such as packet loss, packet reassembly, and time crosstalk. Wang et al. [21] proposed a watermarking scheme based on IPD, which works better than the passive time-based flow association technique with sufficient redundancy. Houmansadr et al. [22] proposed RAINBOW to improve the invisibility of the watermark by introducing lower packet delay than other work.

The packet interval-based flow watermarking techniques [23,24] embed information by modifying the number of packets in a specific interval. The ICBW proposed by Wang et al. [25] uses the uniform distribution principle to embed watermark information by changing the IPD. Pyun et al. proposed the IBW (Interval-Based Watermarking) [26] by modifying the IPD to control the number of packets in four intervals, which has a strong anti-interference ability. The IBF proposed by Luo et al. can embed more fingerprint data with fewer intervals and packets. Houmansadr et al. [27] designed a scalable watermark SWIRL, which selects different watermarks for marking according to flow characteristics, and improves the robustness of the watermark.

Besides, there are some other flow watermarking techniques. The packet payload-based flow watermarking techniques embed information by modifying the content of packets, for example, SWT (Sleepy Watermark Tracing) [28]. The packet length-based flow watermarking techniques embed information by modifying the length of packets, for example, LBW (Length-Based Watermarking) [29]. The packet order-based flow watermarking techniques change the sequence number of packets to embed information, for example, PROFW [30]. The traffic rate-based flow watermarking techniques modify the traffic rate of flow to embed information, for example, PN-based SS (PN Code Diversification Based Spread Spectrum) [31].

We select three aspects to evaluate the performance of several flow watermarking techniques [32]: robustness, invisibility and coding efficiency. Robustness refers to the ability to keep the original data when facing network delays, packet loss and other disturbances. Invisibility refers to the ability that the watermark is invisible to the outside world and can be undiscovered by attackers. Coding efficiency reflects the amount of watermark information embedded in a fixed time unit.

Table 2. Evaluation of several flow watermarking techniques.

Classification	Example	Robustness	Invisibility	Coding Efficiency
Packet payload-based	SWT	★★★	★	★★
Traffic rate-based	PN-based SS	★★	★	★
Packet timing-based	IPD	★	★	★★★
	RAINBOW	★	★★	★★★
Packet interval-based	ICBW	★★	★	★
	IBW	★★	★	★
	IBF	★★	★★	★★
	SWIRL	★★	★★	★
Packet length-based	LBW	★★★	★	★★★
Packet order-based	PROFW	★★★	★★★	★★

★ worse, ★★ normal, ★★★ better.

shows the evaluation of several flow watermarking techniques.

2.3. eBPF

The eBPF [33] is a technique that can run sandboxed programs in the kernel without requiring higher kernel source code or loading kernel modules; it uses a custom 64-bit RISC instruction set and can run just-in-time natively compiled BPF programs in the Linux kernel to make the Linux kernel programmable. There are many open source projects based on eBPF, such as Katran, bcc, bpftrace, cilium, Falco, etc.

The eBPF has a wide range of application scenarios. In the field of tracking and analysis, eBPF can monitor the runtime behavior of applications and the system kernel to provide introspection capabilities for applications and systems. And combine the two views to solve system performance problems. In the security world, eBPF can combine aspects of system call filtering, network-level filtering, and in-city context tracking to create security systems that operate on more contexts and have a better level of control. In the network domain, eBPF allows adding additional protocol parsers and easily tailoring any forwarding logic to meet the needs of SDN.

3. Network Congestion Detection Model Based on Flow Watermarking

3.1. Congestion Detection Model Based on Watermark

The network congestion detection model based on flow watermarking is shown in Figure 2, which mainly includes three logical layers: data layer, codec layer and management layer. When a packet reaches the server core, it enters the data layer. The eBPF extracts the header field of the packet for multi-flow identification. The quintuple information of the encoded and decoded flow distributed by the management layer is used as the key to match. If the match is successful, it indicates that the packet belongs to the encoded flow, which enters the codec layer. Otherwise, it indicates that the packet belongs to the non-encoded flow, which is forwarded directly.

The codec layer includes a watermark encoding module and a watermark decoding module. The watermark encoding module is responsible for embedding the watermark into the encoded flow. According to the watermark distributed by the management layer, eBPF changes the IPD by delaying the forwarding time of the packets or directly forwarding the packets to encode the watermark information. The watermark decoding module is responsible for decoding watermark data destroyed by network congestion. The eBPF counts the number of packets arriving in the corresponding interval of the decoded flow and performs arithmetic operations to obtain the decoding information of the watermark. The information is then passed to the management layer.

The management layer includes a watermark generation module and a network status analysis module. The watermark generation module generates the watermark data, interval length and quintuple information of encoded and decoded flow in real-time according to the network paths to be detected, detection accuracy and other factors. And it sends them to the codec layer and data layer. The network status analysis module analyzes the information from the watermark decoding module to obtain the network status.

We formally describe the generation process of the watermark. We select an end-to-end path P to detect its congestion status. The accuracy of detection is denoted as x, and the unit is millisecond or microsecond. The duration of detection is denoted as T. We select the flow f passing through the path P. The PPS (Packet Per Second) of f is denoted as R(f), and the quintuple identifier is denoted as FlowID. The length of interval is denoted as t, and its value range is:

$$\frac{n}{R\left(f\right)}\le t\le x,$$

(1)

where n is the minimum value of the number of packets in the interval, which can be set to 50, 100, etc.

T and t determine the bits of watermark N, that is

$$N=g\left(\frac{T}{t}\right),$$

(2)

where the function g is determined by a specific watermark encoding method.

According to N and FlowID, the watermark is obtained. To ensure the randomness of watermark generation, random number τ is introduced. The watermark generation function is

$$Watermark=F\left(N,FlowID,\tau \right).$$

(3)

Then network congestion detection model can be defined as:

$$Network Status=Model\left(Watermark,Traffic\right)$$

(4)

where the input is Watermark and Traffic. Traffic represents the distribution of size and arrival time of flows, which may cover different traffic models, such as micro-burst flow model, elephant flow model, etc. The output is Network Status, which indicates the current network status. According to the fine-grained network congestion detection algorithm, it can be divided into normal, slightly congested, moderately congested and heavily congested. Section 5 introduces the algorithm in detail.

3.2. Watermark Mechanism Based on Sparsity of Packet

The flow watermarking based on sparsity of packet uses four consecutive intervals to encode two bits of watermark data, which modifies the number of packets in the middle two intervals and compares it with the average number of packets in the four intervals, that is, the sparsity of packet.

We divide flow F (with a series of packets) into intervals Ii (i < 0, slot index) of length T (>0) starting at random time offset o (>0) and choose four consecutive intervals. The first to fourth intervals are denoted as I_1,k, I_2,k, I_3,k, and I_4,k (k = 1, 2, …, r), where the number of packets contained is expressed as X_1,k, X_2,k, X_3,k and X_4,k (k = 1, 2, …, r). For flows that are sufficiently long duration and contain many intervals, X_j,k (j = 1, 2, 3, 4; k = 1, 2, …, r) satisfy random sample drawn from a sufficiently large general distribution. Therefore, X_j,k satisfy the iid (independent and identical distribution). Assume

$$E\left(X_{j,k}\right)={\mu }_x,V\left(X_{j,k}\right)={\sigma }_x^2.$$

(5)

Given four continuous intervals, the difference between the number of packets in the second interval and the mean number of packets is defined as:

$$Y_{k2}=X_{2,k}-\overline{X_k},$$

(6)

where

$$\overline{X_k}=\frac{1}{4}\left(X_{1,k}+X_{2,k}+X_{3,k}+X_{4,k}\right).$$

(7)

Therefore

$$E\left(\overline{X_k}\right)={\mu }_x,V\left(\overline{X_k}\right)={\sigma }_x^2/4,$$

(8)

$$E\left(Y_{k2}\right)=0,V\left(Y_{k2}\right)\le 5{\sigma }_x^2/4.$$

(9)

Similarly

$$Y_{k3}=X_{3,k}-\overline{X_k},E\left(Y_{k3}\right)=0,V\left(Y_{k3}\right)\le 5{\sigma }_x^2/4.$$

(10)

Because X_2,k satisfy the iid, so do Y_k2 and Y_k3.

To improve the accuracy of watermark codec and avoid the influence of accidental factors, redundant coding r (>0) is introduced, that is, r shares of four consecutive intervals are used to embed two bits of watermark. Thus, the mean of sampling Y_k is defined as

$$\overline{Y_r}=\frac{1}{r}\sum _{k=1}^r{Y}_k.$$

(11)

Since Y_k (k = 1; 2; …; r) satisfy the iid, then

$$E\left({\overline{Y}}_r\right)=0, V\left(\overline{Y_r}\right)\le \frac{5{\sigma }_x^2}{4r}.$$

(12)

Y_k is symmetric about mean 0, so is$\overline{Y_r}$.

According to the watermark data, the watermark coding module directly forwards or delays packets in the specific interval to embed information. Figure 3 shows the schematic diagram of the watermark encoding. When redundant coding is not used (i.e., r = 1), four consecutive intervals are used to encode the two bits of watermark. By delaying the forwarding time of packets, the Y_k2 and Y_k3 are increased or decreased to be larger or smaller than the average number of packets in four intervals to encode four watermarks.

When the watermark ‘00’ is encoded, the packets in the second interval are delayed to the third interval. At the third interval, the delayed packets are sent first. Then the packets arriving this interval are sent. The operation lets Y_k2 decrease by ${\mu }_x$, and Y_k3 increase by ${\mu }_x$. At this time, Y_k2 is less than the average number of packets in four intervals, and Y_k3 is greater than the average value. The encoding operations of other three watermarks are similar.

4. Theoretical Analysis of Network Congestion Detection Based on Flow Watermarking

4.1. Analysis of the SRW without Interference

We define the SRW(success rate of the watermark) as the probability that the two bits of watermark are decoded correctly. That is, when f = 00, the SRW is expressed as $Pr[(\overline{Y_{r2}^f}<0)(\overline{Y_{r3}^f}>0)]$; when f = 01, the SRW is $Pr[(\overline{Y_{r2}^f}\ge 0)(\overline{Y_{r3}^f}<0)]$; when f = 10, the SRW is $Pr[(\overline{Y_{r2}^f}<0)(\overline{Y_{r3}^f}\le 0)]$; when f = 11, the SRW is $Pr[(\overline{Y_{r2}^f}\ge 0)(\overline{Y_{r3}^f}\ge 0)]$. Y_k are defined as $Y_k^f$ (k = 1; 2; …; r) after embedding watermark f. Since $\overline{Y_r}$is symmetrical about the mean value 0, only one case needs to be analyzed. Take $Pr[(\overline{Y_{r2}^f}\ge 0)(\overline{Y_{r3}^f}<0)]$ when f = 01 as an example.

Y_k satisfy random sampling, so do $Y_k^f$. According to the central limit theorem, in the case of random sampling with a large sample, $\overline{Y_{r2}^f}$ and $\overline{Y_{r3}^f}$ can be considered to satisfy the standard normal distribution; thus, the distribution of probability satisfies:

$$Pr\left[\frac{\left(\overline{Y_{r2}^f}-E\left(\overline{Y_{r2}^f}\right)\right)}{\sqrt{V\left(\overline{Y_{r2}^f}\right)}}\ge \xi \right]\approx 1-\Phi \left(\xi \right), Pr\left[\frac{\left(\overline{Y_{r3}^f}-E\left(\overline{Y_{r3}^f}\right)\right)}{\sqrt{V\left(\overline{Y_{r3}^f}\right)}}<\xi \right]\approx \Phi \left(\xi \right),$$

(13)

where

$$\Phi \left(\xi \right)={\int }_{-\infty }^{\xi }\frac{1}{\sqrt{2\pi }}{e}^{-{\mu }^2/2}du$$

(14)

After embedding watermark ‘01’, X_2,k and X_3,k are defined as $X_{2,k}^f$ and $X_{3,k}^f$, namely

$$X_{2,k}^f=X_{1,k}+X_{2,k},X_{3,k}^f=0$$

(15)

From Formula (5) we can obtain:

$$E\left(X_{2,k}^f\right)=2{\mu }_x,E\left(X_{3,k}^f\right)=0$$

(16)

$$V\left(X_{2,k}^f\right)={\sigma }_x^2+{\sigma }_x^2+2Corr\left(X_{1,k},X_{2,k}\right){\sigma }_x^2,V\left(X_{3,k}^f\right)=0$$

(17)

The maximum variance occurs when $Corr\left(X_{1,k},X_{2,k}\right)=1$.

Therefore,

$$E\left(\overline{Y_{r2}^f}\right)={\mu }_x,V\left(\overline{Y_{r2}^f}\right)\le 35{\sigma }_x^2/8r.$$

(18)

$$E\left(\overline{Y_{r3}^f}\right)=-{\mu }_x,V\left(\overline{Y_{r3}^f}\right)\le 3{\sigma }_x^2/8r$$

(19)

From Formula (13) we can obtain:

$$P\left[\overline{Y_{r2}^f}\ge 0\right]=Pr\left[\frac{\left(\overline{Y_{r2}^f}-E\left(\overline{Y_{r2}^f}\right)\right)}{\sqrt{V\left(\overline{Y_{r2}^f}\right)}}\ge \frac{-E\left(\overline{Y_{r2}^f}\right)}{\sqrt{V(\overline{Y_{r2}^f}})}\right]\approx 1-\Phi \left(\frac{-E\left(\overline{Y_{r2}^f}\right)}{\sqrt{V\left(\overline{Y_{r2}^f}\right)}}\right)\ge 1-\Phi \left(\frac{-\sqrt{8r}{\mu }_x}{\sqrt{35}{\sigma }_x}\right).$$

(20)

Similarly,

$$P\left[\overline{Y_{r3}^f}<0\right]=Pr\left[\frac{\left(\overline{Y_{r3}^f}-E\left(\overline{Y_{r3}^f}\right)\right)}{\sqrt{V\left(\overline{Y_{r3}^f}\right)}}<\frac{-E\left(\overline{Y_{r3}^f}\right)}{\sqrt{V(\overline{Y_{r3}^f}})}\right]\approx \Phi \left(\frac{-E\left(\overline{Y_{r3}^f}\right)}{\sqrt{V\left(\overline{Y_{r3}^f}\right)}}\right)\ge \Phi \left(\frac{\sqrt{8r}{\mu }_x}{\sqrt{3}{\sigma }_x}\right).$$

(21)

The SRW without interference is

$$P[(\overline{Y_{r2}^f}\ge 0)(\overline{Y_{r3}^f}<0)]\ge (1-\Phi (\frac{-\sqrt{8r}{\mu }_x}{\sqrt{35}{\sigma }_x}))·\Phi (\frac{\sqrt{8r}{\mu }_x}{\sqrt{3}{\sigma }_x})$$

(22)

It can be concluded that the SRW without external interference increases with the increase of u_x, and decreases with the increase of ${\sigma }_x$.

4.2. Analysis of the SRW When Packet Delay Occurs

Figure 4 shows an example of the distortion of watermark ‘00’ when packet delay occurs. If the packets of the first interval are delayed to arrive at the second interval, and the packets of the third interval are delayed to arrive at the fourth interval. The decoder obtains that Y_k2 is greater than the average value and Y_k3 is less than the average value. The decoded watermark is ‘01’. Similarly, when packets carrying different watermarks face different delays, the decoder also obtains different error results.

The following is a theoretical analysis of the SRW when packet delay occurs, and take $Pr[(\overline{Y_{r2}^f}\ge 0)(\overline{Y_{r3}^f}<0)]$ when f = 01 as an example.

After the packet delay occurs, $X_{i,k}^f$ (i = 1, 2, 3, 4), $\overline{Y_{r2}^f}$ and $\overline{Y_{r3}^f}$ are defined as $X_{i,k}^F$ (i = 1, 2, 3, 4), $\overline{Y_{r2}^F}$ and $\overline{Y_{r3}^F}$. Let $D_{2,k}$ be the number of packets that $X_{2,k}^f$ reduced and define ${\mu }_{d2}$ and ${\sigma }_{d2}$ as the mean and variance of $D_{2,k}$, respectively. Let $D_{3,k}$ be the number of packets that $X_{3,k}^f$ increased and define ${\mu }_{d3}$ and ${\sigma }_{d3}$ as the mean and variance of $D_{3,k}$, respectively. For network with many flows, the packet delays introduced by network congestion or delay jitter satisfy randomness drawn from a general distribution with a sufficiently large sample, so $D_{2,k}$ and $D_{3,k}$ satisfy iid. We have

$$X_{2,k}^F=X_{2,k}^f-D_{2,k},X_{3,k}^F=X_{3,k}^f+D_{3,k},\overline{X_k^F}=\frac{1}{4}\left(X_{1,k}^F+X_{2,k}^F+X_{3,k}^F+X_{4,k}^F\right)$$

(23)

Therefore,

$$E\left(X_{2,k}^F\right)=2u_x-u_{d2},E\left(X_{3,k}^F\right)=u_{d3}$$

(24)

$$V\left(X_{2,k}^F\right)\le 4{\sigma }_x^2+{\sigma }_{d2}^2,V\left(X_{3,k}^F\right)={\sigma }_{d3}^2$$

(25)

The maximum occurs when $Corr\left(X_{2,k},D_{2,k}\right)=0$.

It follows that,

$$E\left(\overline{Y_{r2}^F}\right)=\frac{1}{4}\left(4u_x-3u_{d2}-u_{d3}\right),V\left(\overline{Y_{r2}^F}\right)\le \frac{1}{16r}\left(70{\sigma }_x^2+17{\sigma }_{d2}^2+{\sigma }_{d3}^2\right)$$

(26)

$$E\left(\overline{Y_{r3}^F}\right)=\frac{1}{4}\left(u_{d2}+3u_{d3}-4u_x\right),V\left(\overline{Y_{r3}^F}\right)\le \frac{1}{16r}\left(6{\sigma }_x^2+{\sigma }_{d2}^2+17{\sigma }_{d3}^2\right)$$

(27)

Using the central limit theorem, we obtain:

$$P\left[\overline{Y_{r2}^F}\ge 0\right]\approx 1-\Phi \left(\frac{-E\left(\overline{Y_{r2}^F}\right)}{\sqrt{V\left(\overline{Y_{r2}^F}\right)}}\right)\ge 1-\Phi \left(\frac{-\sqrt{r}\left(4{\mu }_x-3u_{d2}-u_{d3}\right)}{\sqrt{70{\sigma }_x^2+17{\sigma }_{d2}^2+{\sigma }_{d3}^2}}\right).$$

(28)

Similarly

$$P\left[\overline{Y_{r3}^F}<0\right]\approx \Phi \left(\frac{-E\left(\overline{Y_{r3}^F}\right)}{\sqrt{V\left(\overline{Y_{r3}^F}\right)}}\right)\ge \Phi \left(\frac{\sqrt{r}\left(u_{d2}+3u_{d3}-4u_x\right)}{\sqrt{6{\sigma }_x^2+{\sigma }_{d2}^2+17{\sigma }_{d3}^2}}\right).$$

(29)

The SRW when packet delay occurs is

$$P\left[\left(\overline{Y_{r2}^F}\ge 0\right)\left(\overline{Y_{r3}^F}<0\right)\right]\ge (1-\Phi (\frac{-\sqrt{r}\left(4{\mu }_x-3u_{d2}-u_{d3}\right)}{\sqrt{70{\sigma }_x^2+17{\sigma }_{d2}^2+{\sigma }_{d3}^2}}))·\Phi (\frac{\sqrt{r}\left(4u_x-u_{d2}-3u_{d3}\right)}{\sqrt{6{\sigma }_x^2+{\sigma }_{d2}^2+17{\sigma }_{d3}^2}})$$

(30)

Obviously, the SRW when packet delay occurs is significantly lower than the SRW without any interference. Because of packet delay, the difference between the number of packets in second interval and the average number in four intervals decreases, and the variance increases. Similarly, the difference between the number of packets in third interval and the average number increases, and the variance also increases. The greater the change in the number of packets, the lower the SRW; thus, the change in SRW can reflect the packet delay in the network.

4.3. Analysis of the SRW When Packet Loss Occurs

Figure 5 shows an example of the distortion of watermark ‘00’ when packet loss occurs. When the packets of the third interval are lost, the decoder counts the number of packets, and finds that Y_k2 and Y_k3 are less than the average value. The decoded watermark is ‘10’. Similarly, when packets carrying different watermarks face different packet loss, the decoder also obtains different error results.

Taking $Pr[\left(\overline{Y_{r2}^P}\ge 0\right)(\overline{Y_{r3}^P}<0)]$ when f = 01 as an example to analyze the SRW when packet loss occurs. After packet loss occurs, $X_{i,k}^f$(i = 1, 2, 3, 4), $\overline{Y_{r2}^f}$ and $\overline{Y_{r3}^f}$ are defined as$X_{i,k}^P$ (i = 1, 2, 3, 4), $\overline{Y_{r2}^P}$ and $\overline{Y_{r3}^P}$. Let $P_{2,k}$ be the number of packets that $X_{2,k}^f \mathrm{reduced}$, and define ${\mu }_{p2}$ and ${\sigma }_{p2}$ as the mean and variance of $P_{2,k}$, respectively; it is obvious that $P_{2,k}$ satisfy iid. We have

$$X_{2,k}^P=X_{2,k}^f-P_{2,k},X_{3,k}^P=X_{3,k}^f,\overline{X_k^P}=\frac{1}{4}\left(X_{1,k}^P+X_{2,k}^P+X_{3,k}^P+X_{4,k}^P\right)$$

(31)

Then

$$E\left(X_{2,k}^P\right)=2u_x-u_{d2},V\left(X_{2,k}^P\right)\le 4{\sigma }_x^2+{\sigma }_{d2}^2$$

(32)

$$E\left(X_{3,k}^P\right)=0,V\left(X_{3,k}^P\right)=0.$$

(33)

It follows that

$$E\left(\overline{Y_{r2}^P}\right)=\frac{1}{4}\left(4u_x-3u_{d2}\right),V\left(\overline{Y_{r2}^P}\right)\le \frac{1}{16r}\left(70{\sigma }_x^2+17{\sigma }_{d2}^2\right)$$

(34)

$$E\left(\overline{Y_{r3}^P}\right)=\frac{1}{4}\left(u_{d2}-4u_x\right),V\left(\overline{Y_{r3}^P}\right)\le \frac{1}{16r}\left(6{\sigma }_x^2+{\sigma }_{d2}^2\right)$$

(35)

Using the central limit theorem, we obtain:

$$P\left[\overline{Y_{r2}^P}\ge 0\right]\approx 1-\Phi \left(\frac{-E\left(\overline{Y_{r2}^P}\right)}{\sqrt{V\left(\overline{Y_{r2}^P}\right)}}\right)\ge 1-\Phi \left(\frac{-\sqrt{r}\left(4u_x-3u_{d2}\right)}{\sqrt{70{\sigma }_x^2+17{\sigma }_{d2}^2}}\right).$$

(36)

Similarly

$$P\left[\overline{Y_{r3}^P}<0\right]\approx \Phi \left(\frac{-E\left(\overline{Y_{r3}^P}\right)}{\sqrt{V\left(\overline{Y_{r3}^P}\right)}}\right)\ge \Phi \left(\frac{\sqrt{r}\left(u_{d2}-4u_x\right)}{\sqrt{6{\sigma }_x^2+{\sigma }_{d2}^2}}\right).$$

(37)

The SRW when packet loss occurs is

$$P\left[\left(\overline{Y_{r2}^F}\ge 0\right)\left(\overline{Y_{r3}^F}<0\right)\right]\ge (1-\Phi (\frac{-\sqrt{r}(4u_x-3u_{d2})}{\sqrt{70{\sigma }_x^2+17{\sigma }_{d2}^2}}))·\Phi (\frac{\sqrt{r}(4u_x-u_{d2})}{\sqrt{6{\sigma }_x^2+{\sigma }_{d2}^2}})$$

(38)

Compared with the SRW without any interference, the SRW when packet loss occurs is lower. The greater the packet loss rate, the greater the change in number of packets in the same interval. And the greater the change of variance and mean, the lower the SRW; thus, the change of the SRW can roughly reflect the packet loss in the network.

5. Fine-Grained Network Congestion Detection Algorithm Based on Flow Watermarking

There is micro-burst traffic in the Internet and datacenter network. The emergence of burst traffic may cause queuing in switch buffers or even packet loss, which has a great impact on network performance. Therefore, how to reduce the damage of micro-burst traffic to network performance is very important. The primary task is to detect network latency and packet loss caused by micro-burst traffic in real-time.

5.1. Analysis of SRW in Micro-Burst Traffic Model

Figure 6 shows an example of the decoding result of watermark ‘00’ when micro-packet delay and micro-packet loss occurs. If some packets in the first interval are delayed until the second interval, the decoder obtains that Y_k2 is less than the average value and Y_k3 is greater than the average value. The decoded watermark is ‘00’. Similarly, if some packets in the third interval are lost, Y_k2 is less than the average value and Y_k3 is greater than the average value. The decoded watermark is still ‘00’. Therefore, when slight packet delay and packet loss occurs, the watermark decoding result may be deceptive due to the small change in the number of packets, which will affect the judgment of the changes in network status.

5.2. Fine-Grained Network Congestion Detection Algorithm

The fine-grained network congestion detection algorithm is based on the flow watermarking, and it judges the changes in network status according to the difference between the proportion of the number of packets in a certain interval and the ideal proportion of the number of packets in the interval, that is, the Crps (Change ratio of sparsity of packet). Since the delay operations are different for different codes in different intervals, the Crps of four codes need to be analyzed in detail.

After a small packet delay or packet loss occurs, $X_{i,k}^f$ (i = 1, 2, 3, 4), $\overline{Y_{r2}^f}$ and $\overline{Y_{r3}^f}$ are defined as$X_{i,k}^N$ (i = 1, 2, 3, 4), $\overline{Y_{r2}^N}$ and $\overline{Y_{r3}^N}.$ Let $N_j$ (j = 0, 1, 2, 3) be the number that $X_{i,k}^f$ increase or decrease when encoding ‘j’, and $n_j$ (j = 0, 1, 2, 3) be the proportion of the number that $X_{i,k}^f$ change. For different codes, $N_j$ and $n_j$ are analyzed in different intervals.

For the code ‘00’, without external interference, $X_{2,k}^f=0,X_{3,k}^f=2u_x$. In realistic scenes, if some packets in the third interval are congested until the fourth interval or some packets in the third interval are lost, it can be inferred that when $N_0$ is less than u_x, network congestion has occurred, but the correct result can still be obtained. We have

$$X_{3,k}^N-u_x=n_0\ast u_x (0<n_0<100\%),$$

(39)

$$n_0=(X_{3,k}^N-u_x)/u_x (0<n_0<100\%).$$

(40)

At this time $\overline{Y_{r3}^N}\ge 0$. The smaller the n₀, the greater the change of sparsity of packet. At this time, the greater the changes in network status, correspondingly. Therefore, when the decoding result is ‘00’, n₀ can reflect the small packet delay and packet loss, which is more helpful for the analysis of the change in network status.

The analysis of the code ‘00’ is analogized to other codes. For code ‘01’, ideally, $X_{2,k}^f=2u_x, X_{3,k}^f=0$. In realistic scenes, some packets in the second interval are congested until the third interval, or some packets in the second interval are lost; it can be inferred that when $N_1$ is less than u_x, network congestion has occurred, but the correct result can still be obtained. We have

$$X_{2,k}^N-u_x=n_1\ast u_x 0<n_1<100\%,$$

(41)

$$n_1=(X_{2,k}^N-u_x)/u_x 0<n_1<100\%$$

(42)

The smaller n₁, the greater the change of sparsity of packet.

For code ‘10’, ideally, $X_{2,k}^f=0, X_{3,k}^f=0, X_{4,k}^f=3u_x$. In realistic scenes, some packets in the fourth interval are congested until the next interval, or some packets in the fourth interval are lost; it can be inferred that when $N_2$ is less than 2u_x, network congestion has occurred, but the correct result can still be obtained. We have

$$3u_x-X_{4,k}^N=n_2\ast u_x (0<n_2<200\%),$$

(43)

$$n_2=(3u_x-X_{4,k}^N)/u_x (0<n<200\%).$$

(44)

The larger n₂, the greater the change in sparsity of packet.

For code ‘11’, ideally, $X_{2,k}^f=1.5u_x, X_{3,k}^f=1.5u_x$. In realistic scenes, some packets in the third interval are congested until the fourth interval, or some packets in the third interval are lost; it can be inferred that when $N_3$ is less than 0.5u_x, network congestion has occurred, but the correct result can still be obtained. We have

$$X_{3,k}^N-u_x=n_3\ast u_x (0<n_3<50\%),$$

(45)

$$n_3=(X_{3,k}^N-u_x)/u_x (0<n_3<50\%).$$

(46)

The smaller the n₃, the greater the change of sparsity of packet.

We define the DRW(detection rate of the watermark) as the percentage of the number of correctly identified bits of the watermark in the total number of encoded bits of the watermark. Let $\overline{n_j}\left(j=0, 1, 2, 3\right)$be the mean of multiple$n_j$. For the encoded flow f, the watermark decoding module performs the corresponding operation to obtain $n_j$ according to the four decoded results. Comparing the decoded value of watermark with the encoded value of watermark, the DRW is obtained. According to the DRW, the network status can be first divided into four levels: normal, slightly congested, moderately congested and heavily congested. When the network status are classified as normal, the $\overline{n_j}$ corresponding to the correctly decoded watermark are calculated, respectively. The Crps is used to detect micro-burst traffic. The changes in network status are divided into more fine-grained levels to realize network congestion detection on a small time scale.

6. Evaluation

6.1. Experiment Environment

The experimental network topology is shown in Figure 7, which are three scenes. Each scene consists of PCs running Ubuntu 18.04 and routers. The network congestion detection model designed in Section 3 is implemented on the PC based on eBPF, and the router runs the BGP routing protocol to achieve network interconnection. We use iPerf to generate flow with a specified bandwidth, and the Traffic Control module to set the link packet delay and packet loss rate.

Figure 7a is an ideal scene without any interference flow. C1 sends the encoded flow f. The C2–C3 link controls the packet delay and packet loss rate. C4 receives the packets belong to f and decodes watermark. Figure 7b is a realistic scene when bandwidth is the bottleneck. C5 sends interference flows of different sizes to C6. The encoded flow and the interference flow share the transmission path of C2–C3. C4 decodes and analyzes the changes in the encoded flow with interference flows. Figure 7c is a realistic scene when process performance is the bottleneck. C5 sends interference flows of different sizes to C4. The encoded flow and the interference flow share the transmission path of C2–C4 and the decoder port of C4. C4 decodes and analyzes the changes in the encoded flow with interference flows.

6.2. Experiment Result and Analysis

(1)

The increase of the length of interval and flow rate will improve DRW.

DRW greatly affects the judgment of network status. In Figure 7a, we set the packet loss rate as 5%, the average delay jitter as 20 ms and the flow rate as 1 M/s, 10 M/s and 20 M/s in different interval length to evaluate the detection rate of watermark based on sparsity of packet. As shown in Figure 8, the increase of the length of interval and the flow rate will improve DRW; thus, we set length of interval as 500 ms and flow rate as 10 M/s to evaluate the performance of network congestion detection based on flow watermarking.

(2)

DRW in different packet delays and packet loss rates.

In order to evaluate the detection ability of watermark in different packet delays and packet loss rates, we evaluate DRW in three scenes.

In Figure 7a, we set the average delay jitter as 0~500 ms, the length of interval as 500 ms and the average packet loss rate as 0~20%. From Figure 9, it can be concluded that with the increase of packet delay and packet loss rate, the DRW of three watermarks decrease accordingly. Compared with the other two watermarks, the watermark based on sparsity of packet has a larger change, which can better reflect the changes in network status.

In order to make evaluation in line with real network status, we simulate a more real network environment by adding interference flows to create congestion.

In Figure 7b,c, we set the length of interval as 500 ms, the flow rate of the encoded flow as 10 M/s, the limit rate of shared link C2–C3 as 20 M/s, and the rate of the interference flow as 5 M/s, 10 M/s, 15 M/s, and 20 M/s, respectively, and evaluate the DRW of the encoded flow in C4. When the rate of interference flow increases gradually, the packet loss rate of the encoded flow increases accordingly. As shown in Figure 10, as the packet loss rate of the encoded flow increases, the DRW decreases. In Figure 7c, because the encoded flow and the interference flow share the transmission path of C2–C4 and the decoder port of C4, the DRW is slightly lower than the DRW in Figure 7b.

It be concluded that the watermark will change when network packet delay and packet loss occur. The greater the packet delay and packet loss rate, the lower the DRW; however, the change of the DRW is small when the changes in network status is slight. For example, in Figure 7a, when the packet loss rate is 1% and 5%, the DRW is about 93%, which is easy to cause confusion in the analysis of the changes in network status. Therefore, it is necessary to combine the fine-grained network congestion detection algorithm for further judgment. We evaluate the performance of fine-grained network congestion detection algorithm in three scenes.

(3)

Evaluation of fine-grained network congestion detection algorithms in three scenes.

In Figure 7a, we set the length of interval as 500 ms and the average delay jitter as 0~200 ms to evaluate the Crps of four codes. Figure 11 shows the Crps in different packet delays; it can be concluded that with the increase of packet delay, the Crps change accordingly, and there is a certain range of change when small packet delay occurs. Compared with the DRW, the Crps have larger variation. During the packet delay of 200 ms, the DRW changes by 20%, but the average Crps change by 50%; thus, the Crps can better show the changes in network status.

In Figure 7b, we set the length of interval as 500 ms, the rate of the encoded flow as 10 M/s, the limited rate of shared link C2–C3 as 20 M/s, and the rate of interference flow as 5 M/s, 10 M/s, 15 M/s, 20 M/s, respectively, to evaluate the Crps of four codes in C4. As shown in Figure 12, the same conclusion can be drawn. As the packet loss rate increases, the Crps change accordingly, and there are certain changes when small packet loss occurs. Compared with the DRW, some Crps have a greater variation. When the packet loss rate is 45%, the DRW changes by about 55%, but the n₂ changes by about 65%. The n₀ has changed by about 58% when the packet loss rate is 37.85%; thus, some Crps can better show the changes in network status.

In Figure 7c, we set the length of interval as 500 ms, the rate of the encoded flow as 10 M/s, the limited rate of shared link C2 –C3 as 20 M/s, and the rate of interference flow as 5 M/s, 10 M/s, 15 M/s, 20 M/s, respectively, to evaluate the Crps of four codes in C4. As shown in Figure 13, the same conclusion can still be drawn. As the packet loss rate increases, the Crps change accordingly, and there are certain changes when small packet loss occurs. Compared with the DRW, some Crps have a greater variation; thus, some Crps can better show the changes in network status.

It can be concluded that with the increase of the packet delay and packet loss rate, the four Crps change accordingly, and there is a certain range of change when slight congestion occurs. Using the Crps to judge the changes in network status can obtain more accurate network information. Therefore, the fine-grained network congestion detection algorithm based on flow watermarking can reflect the small change of network status and realize more accurate network congestion detection.

7. Conclusions and Further Work

Facing the problems of occupying extra bandwidth, decreasing the ratio of the effective payload of the packet, and increasing the burden of the switches in the current network congestion detection, this paper proposes a fine-grained network congestion detection method based on flow watermarking, and combines it with the eBPF, which realizes fine-grained efficient network congestion detection with low overhead and easy deployment.

This paper first proposes a network congestion detection model based on flow watermarking. Then, the theoretical analysis of network congestion detection based on flow watermarking is carried out. And a fine-grained network congestion detection algorithm is designed for the problem that the congestion caused by the micro-burst traffic is difficult to detect.

The experimental results based on the eBPF environment show that the changes in network status (normal, congested, etc.) can be obtained in real-time and accurately by analyzing the watermark decoding information. The watermark based on the sparsity of the packet can better reflect the network status and the fine-grained network congestion detection algorithm can achieve congestion detection on a small time scale.

Despite the network congestion detection method based on flow watermarking performing well, there are other challenges that are not addressed. Our future work will investigate the impact of different traffic scheduling algorithms on the network congestion detection method based on flow watermarking. We will also evaluate the effectiveness and overhead of this method in practical implementation.