MonLink: Piggyback Status Monitoring over LLDP in Software-Defined Energy Internet

Abstract

While software-defined networking (SDN) has been widely applied in various networking domains including datacenters, WANs (Wide Area Networks), QoS (Quality of Service) provisioning, service function chaining, etc., it also has foreseeable applications in energy internet (EI), which envisions an intelligent energy industry on the basis of (information) internet. Global awareness provided by SDN is especially useful in system monitoring in EI to achieve optimal energy transportation, sharing, etc. Link layer discovery protocol (LLDP) plays a key role in global topology discovery in software-defined energy internet when SDN is applied. Nevertheless, EI-related status information (power loads, etc.) is not collected during the LLDP-based topology discovery process initiated by the SDN controller, which makes the optimal decision making (e.g., efficient energy transportation and sharing) difficult. This paper proposes MonLink, a piggyback status-monitoring scheme over LLDP in software-defined energy internet with SDN-equipped control plane and data plane. MonLink extends the original LLDP by introducing metric type/length/value (TLV) fields so as to collect status information and conduct status monitoring in a piggyback fashion over LLDP during topology discovery simultaneously without the introduction of any newly designed dedicated status monitoring protocol. Several operation modes are derived for MonLink, namely, periodic MonLink, which operates based on periodic timeouts, proactive MonLink, which operates based on explicit API invocations, and adaptive MonLink, which operates sensitively and self-adaptively to status changes. Various northbound APIs are also designed so that upper layer network applications can make full use of the status monitoring facility provided by MonLink. Experiment results indicate that MonLink is a lightweight protocol capable of efficient monitoring of topological and status information with very low traffic overhead, compared with other network monitoring schemes such as sFlow.

1. Introduction

J. Rifkin first put forward the concept of energy internet (EI) in his work about the third industrial revolution that he believed will happen in the foreseeable future [1]. Five pillars must be constructed before the revolution takes place: (1) renewable energy; (2) micro power plants converted from buildings for energy recycling; (3) hydrogen-based energy storage; (4) energy sharing based on internet technologies; (5) new energy vehicles, which comprehensively pave the foundation for the third industrial revolution.

Energy internet will comprise the backbone and infrastructure for the intelligent energy generation, harvesting, storage, management, transportation, sharing, etc. Similarities are shared between (information) internet and energy internet. While internet has routers to steer traffic between different local area networks (LANs), EI has power routers to steer energy between different power distribution networks (PDNs). Similarly, while the internet has switches to steer traffic inside a LAN, EI has power switches to steer energy inside a PDN. Therefore, facilitating EI on the basis of Internet is a natural choice. However, energy internet has far more complex content than that of internet. While the internet plays a role as the information hub with high-density computational, storage, and network resources, energy internet integrates energy industry, IoT, sensor technologies, big data, low-power WAN, etc., on the basis of internet. Thus simply expanding the internet to the energy field is far from addressing issues faced with EI. Therefore, various EI-related projects have been proposed by different countries such as FREEDM (Future Renewable Electric Energy Delivery and Management) in US [2,3], e-energy in Germany [4], etc.

Energy internet will also extend the information frontier and deepen the information depth beyond the reach of traditional internet. For example, for those remotely located renewable energy plants (such as wind power and photovoltaic power plants) sensitive to weather conditions, system status and electricity outcome are time-variant thus requiring long-term monitoring. Meanwhile, sensors connected to photovoltaic or wind power generators have limited battery thus requiring low-power transmission and well-tuned sampling intervals for long-term surveillance. In addition, for remotely located renewable energy plants, human configuration at site might not be possible due to severe natural conditions. It is a desirable feature if smart power equipments can be discovered (together with their system status information), monitored and controlled by a smart central hub once they are installed possibly through some automated mechanics. To summarize, automatic equipment discovery, low-power adaptive sensing, continuous real-time monitoring of the entire power plant and smart decision making will play key roles for the sustainable operation of energy internet. This requires new capabilities, especially control and management capabilities, to be added to the current internet to facilitate energy internet.

Despite the differences, EI and internet share similarities on structure backbone and networking mechanisms. New advancements in internet can be reasonably borrowed and adapted to EI. Software-defined networking (SDN) [5] is widely adopted in communication networks such as WAN (Wide Area Network) [6,7], datacenters [8,9,10,11], NFV (Network Function Virtualization) [12], network virtualization [13,14], service provisioning [15,16,17,18,19,20]. SDN introduces logically centralized software controller with elaborate control functions to enable smarter traffic steering, policy enforcement, global management over the whole network, etc. Therefore, methodologies in SDN can be applied in EI to control intelligent sensors, power routers/switches, thus a software-defined energy internet with SDN-equipped control plane and data plane that is capable of intelligent energy manipulation.

SDN was originally proposed by Stanford University to support the design of a network innovation platform based on the OpenFlow protocol [21,22], an open protocol that aims to break down vendor-specific barriers to facilitate network innovation. Among others, OpenFlow is the most accepted southbound interface (i.e., the controller-switch interface) for the controller to manage underlay switches that are OpenFlow-compatible. OpenFlow reveals the programmability of network devices, especially switches, by providing standardized programming interfaces. In addiction to the powerful programmability and flexible architecture, SDN is also equipped with the global topological view based on which globally optimized network resource utilization and decision making become possible. Link layer discovery is the foundation for the retrieval of global topological view and related information. The IEEE-proposed link layer discovery protocol (LLDP) [23] is the primary link layer discovery protocol used by SDN controllers. Since SDN works in a centralized fashion, LLDP-collected topological information is aggregated into the centered controller.

Should time-variant equipment metrics be piggybacked by LLDPDUs (Link Layer Discovery Protocol Data Units) traveling around a software-defined energy internet, system status information can be possibly aggregated into the controller during topology discovery to support lightweight and continuous monitoring in an energy internet, without drastic modification to the current system structure. Based on this thought, this paper proposes MonLink, the piggyback monitoring over LLDP for software-defined energy internet, so that the lightweight status-aware topology discovery becomes possible. MonLink will work as the status information monitoring infrastructure that supports various status-aware applications. The contribution of this paper includes as follows:

• MonLink monitors the system status information in a piggyback fashion over LLDP during topology discovery. Since LLDP is widely adopted for topology discovery in various controllers, the usage of LLDP as the fundamental retrieval method for status information is “transparent” to SDN networks including software-defined energy internet.
• MonLink takes advantage of the extensibility of LLDP. It does not introduce any newly designed dedicated status monitoring protocol, thus very little traffic overhead during status monitoring. Therefore, it does not affect or compromise the working process of southbound interface of SDN (e.g., OpenFlow), i.e., a non-invasive design. (see Section 2.2 and Section 2.7)
• MonLink works in a hybrid fashion that comprehensively considers self-adaptiveness and high interactiveness, meeting the requirements of time-variant and continuous monitoring in energy internet. (see Section 2.3, Section 2.4, Section 2.5 and Section 2.6)
• MonLink makes use of the centralized control in SDN. No extra network monitoring system (such as sFlow [24]) is required to implement status information collecting (see Section 3.2).

The rest of this paper is organized as follows: Section 2 explains the protocol extension, design, and workflow of MonLink. It also specifies several working modes of MonLink that fit in different application scenarios to achieve flexible status information collecting. Section 3 compares MonLink with sFlow-based status information collecting through extensive experiments. Related works are summarized in Section 4. Finally, this paper is concluded in Section 5.

2. Materials and Methods

2.1. Software-Defined Energy Internet

In this paper, software-defined energy internet refers to the the SDN-equipped energy internet. Specifically, it is an energy internet equipped with the SDN control plane (i.e., OpenFlow-compatible SDN controllers) and SDN data plane (forwarding devices that support OpenFlow protocol). SDN controllers are easy to be deployed in energy internet since they are independent software entities. On the other hand, the deployment of SDN data plane involves two aspects: (1) the deployment of OpenFlow (power) switches/routers inside a renewable or traditional power plant; and optionally, (2) the enhancement of sensors (temperature sensors, humidity sensors, cameras, etc.) and device controllers (e.g., battery controllers) with OpenFlow agents (e.g., the software-implemented open vSwitch (OVS) [25]) installed inside these sensory terminals. If aspect (2) is additionally implemented, sensors and controllers are also part of the SDN data plane thus can be controlled directly by SDN controllers. This can enable an end-to-end (i.e., from the central SDN controller to edge power generators) controllable smart energy internet, simply using OpenFlow PDUs (Protocol Data Units) such as flow table entries with directive messages (i.e., flow rules) that sensors and controllers should obey. If only aspect (1) is implemented, status information of power generators collected by sensors can still be aggregated by OpenFlow (power) switches, and then sent to SDN controllers that are capable of making smart decisions based on the collected status information, and controlling edge power generators through other dedicated protocols. Either way enhances the controllability and capabilities of smart decision-making in energy internet, although the SDN-controlled coverage is different.

Figure 1 demonstrates the composition of the software-defined energy internet. The right part is a photovoltaic power plant network, i.e., a software-defined energy LAN. Sensors and device controllers are indirectly controlled by the SDN controller by means of OpenFlow protocol through wired (e.g., Ethernet) or wireless (LoRa, NB-IoT, ZigBee, etc.) channels, possibly protected by secure methods [26,27] in case of the need for security. Many such software-defined energy LANs are connected by OpenFlow power routers to constitute the software-defined energy internet (the left part). Applications can be built on top of energy internet by invoking various northbound interfaces (green dashed lines) provided by SDN controllers. Therefore, the key to revealing the full potential of software-defined energy internet is a rich set of SDN-equipped control functions oriented to power devices, which are exposed by corresponding northbound interfaces. The remainder of this paper describes one such control function, MonLink, the piggyback status monitoring over LLDP in software-defined Energy Internet.

2.2. Protocol Extension to LLDP

Although the SDN controller maintains the awareness of the existence of directly connected switches by sending and receiving “hello” messages, it is not aware of the topological structure formed by interconnected switches. To acquire the topological knowledge, the SDN controller must conduct the link layer discovery so as to reveal how underlying switches are mutually connected once the SDN network is initiated. The IEEE-proposed LLDP is the de facto protocol to fulfill this centralized topology discovery in SDN field.

LLDP works along with the flow matching mechanism in SDN. A packet_out message is sent by the controller to one of the connected switches to instruct it to multicast the LLDP packet contained in the packet_out. Inside the LLDP packet, basic information about this switch (such as MAC address) is included. All other switches (i.e., receiver switches) connected to this sender switch receive and match this LLDP packet against their own flow tables. At this point of time, no dedicated flow table entries are installed for LLDP traffic, so that this mis-matched LLDP packet is sent to the controller by means of packet_in to inquire how it should be processed. Since the packet_in contains both the sender and receiver switches’ MAC (Media Access Control) addresses, etc., it is reasonable for the controller to assert that there exists a link between these two switches. This LLDP multicast process instructed by the controller is periodically carried out throughout the lifespan of the SDN network so that the global topology as well as its continuous updates are captured in a timely fashion. If extra status information is somehow included in LLDP packets in addiction to basic information such as MAC addresses, status monitoring can occur simultaneously with topology discovery/maintenance.

This is especially appealing for power device/equipment discovery and continuous monitoring when the SDN methodologies are deployed in the energy internet. The key lies in how status information can be incorporated. Thanks to the extensibility offered by LLDP, status information can be encapsulated inside TLV (Type/Length/Value, i.e., key-value pair with length information) fields. The string value subfield is where extra information resides, as shown in the “Metrics” in Figure 2. Currently, we consider communication-centric properties such as delay, bandwidth, packet loss, and jitter, 8 bytes for each metric. Note that metrics and length assigned for each metric can vary according to application configurations. For example, metrics such as temperature, humidity, battery, etc., can be included for energy internet oriented applications. We call this piggybacked status monitoring on the basis of LLDP MonLink.

In order to achieve flexible status information collection in different application scenarios, several working modes of MonLink were designed: periodic MonLink, which is triggered by timeouts, adaptive MonLink, which is an adaptive version of periodic MonLink, and proactive MonLink, which is triggered by explicit user invocation, to adapt our scheme to different network environments and requirements.

2.3. Periodic MonLink

The periodic MonLink follows the convention of the standard LLDP that LLDPDUs are sent out by the controller on a periodic basis during the life cycle of an SDN network. Once a switch receives an LLDPDU, periodic MonLink is carried out along with the topology discovery. The disadvantage of periodic MonLink is that LLDP multicast interval might not be short enough for scenarios where metrics fluctuate violently and frequently. During the LLDP multicast interval, status database in the controller is not updated. The worst case is that metrics fluctuation happens during the interval (i.e., between two multicasts) and recovers at almost the same time the next multicast happens (usually in a scale of several milliseconds). In this case, the collected status information is almost identical to that collected at the previous round, failing to capture the dynamics and violent metrics fluctuation between two multicasts.

The previous problem can be relieved using reduced LLDP multicast interval. However, smaller interval means more frequent sending of LLDP multicasts which in turn result in heavier traffic overhead caused by MonLink. This would consume larger bandwidth meant for user traffic since the underlay network is an in-band legacy network where overhead traffic including LLDP is transmitted along with user-data traffic in the same data plane. Thus the optimal interval relies on deep analysis of historical trending of status changes so that the LLDP multicast interval can be fine-tuned to better match the status collecting. Therefore, APIs dedicated for interval fine-tuning is provided by MonLink. However, to determine an appropriate multicast interval is still a challenging task, requiring both experience and deep theoretical study. In order to enhance adaptiveness and interactiveness, two more working modes of MonLink are derived.

2.4. Proactive MonLink

In order to cope with the disadvantage faced with periodic MonLink, we specifically designed the proactive MonLink. Individual monitoring requests might come to the SDN network in an ad hoc fashion, thus the SDN controller must be able to acquire the status information or update the previous status database on demand, especially for time-critical scenarios. Periodic MonLink cannot obtain status information on demand. Therefore, proactive MonLink is designed to fulfill the on-demand status information collecting. This working mode offers the chance for proactive interference by network administrators or applications. Therefore, APIs must be given on top of the SDN controller for the network administrators or applications to initiate a new round of status information collecting. Whenever an ad hoc status monitoring request comes, the controller can be automatically triggered to start a new round of MonLink multicast. In a word, proactive MonLink complements periodic MonLink with explicit interference from upper-layer entities (network applications, network administrators, etc.).

2.5. Adaptive MonLink

As mentioned above, to determine the appropriate sampling intervals between any two consecutive MonLink multicasts was non-trivial work. It was a desirable feature if the sampling intervals can be determined automatically and adaptively. To achieve this, MonLink must work in a stateful manner, keeping records of currently collected status information and deciding intervals accordingly by elaborating adaptive algorithms. What was expected was that intervals automatically shrank when status information varied violently in a short term while intervals steadily expand to initial values when status information remained almost unchanged. This required short-term prediction based on the current status. The core of the adaptive interval algorithm is shown in Equation (1).

$$I_i\left(t\right)=I_{base}·e^{-\sqrt{{\sum }_{j=1}^N{\left(W_j·\frac{S_{i,j}\left(t\right)-S_{i,j}(t-1)}{S_{bench,j}}\right)}^2}}+I_{min}$$

(1)

It is a Euclidean distance based decay function that predicts interval of the next MonLink iteration.

Table 1. Notations and definitions for adaptive MonLink.

Notations	Definitions
M	The number of MonLink monitored equipments
N	The dimensionality of status information
$I_{periodic}$	The sampling interval for periodic MonLink
$I_{min}$	The minimum sampling interval for adaptive MonLink
$I_{base}$	The base sampling interval for adaptive MonLink
$I_i\left(t\right)$	The calculated interval for the i-th monitored equipment at iteration t
$IS\left(t\right)$	The set to store all calculated intervals for monitored equipments at iteration t
$I_{mc}\left(t\right)$	The multicast interval for adaptive MonLink
$S_{bench}$	The j-dimensional status benchmark
$S_{bench,j}$	The j-th dimension of $S_{bench}$
$S_i\left(t\right)$	The j-dimensional status information for the i-th monitored equipment at iteration t
$S_{i,j}\left(t\right)$	The j-th dimension of $S_i\left(t\right)$
$W_j$	The weight of the j-th dimension of status information

specifies the notations used in Equation (1) and Algorithm 1 (i.e., the adaptive MonLink). Inside the square root part is the Euclidean distance between the current status information and the status collected at the previous iteration.

The reason why the Euclidean distance was used is due to its tendency irrelevance, meaning that, no matter if a metric increases or decreases, the change can be correctly captured using the same equation. Metrics can be favored using different weights (i.e., $W_j$) for distance computation. Meanwhile, as seen in Equation (1), we did not merely use Euclidean distance; instead, a Euclidean distance-based decay function was also used. The reason why the interval was finally determined by a decay function instead of just the Euclidean distance was that the non-linear decay function varied much faster than the linear Euclidean distance. If changes of status indicated by Euclidean distance were detected at present time, exhibiting the initial trend of system changes, MonLink must get ready for this trend in advance using even smaller intervals determined by the decay function. Therefore, short-term prediction became possible. Once the system status became stable again (i.e., almost unchanged), regardless of the absolute values, the interval automatically increased back to be around maximum (i.e., $I_{base}+I_{min}$) due to that $e^{-d}\to 1$ given $d\to 0$, where d is the Euclidean distance. This is how adaptivity becomes possible based on Equation (1).

Besides, the status value difference $\left(S_{i,j}\left(t\right)-S_{i,j}(t-1)\right)$ is divided by a status benchmark value $S_{bench,j}$ in Equation (1). Therefore, status benchmark settings significantly affected the sensibility of adaptive MonLink. Smaller settings led to an adaptive MonLink which was much more sensitive to minor status changes, due to the non-linearity of the exponential function $e^{-d}$. However, if the benchmark settings are too small (resulting in greater Euclidean distance thus smaller interval), two side-effects might occur: (1) too frequent MonLink multicasts might cause overwhelming LLDP traffic to compete the bandwidth; (2) the interval is so small that there might not be enough time for MonLink multicasts to take effect. Therefore, we elaborately provide the minimum interval $I_{min}$ as the additive factor (the immutable part) in Equation (1) to ensure MonLink multicasts work properly to successfully collect status information. Presently, $I_{min}$ is set to one second. The mutable part of the interval was determined by the decayed base interval $I_{base}$ to capture the system variation, where $I_{base}=I_{periodic}-I_{min}=14$. Currently, we manually configure the status benchmark settings according to experience. Nevertheless, it requires expertise in understanding and tuning status benchmark settings. Even an experienced network administrator can hardly give settings that fully capture the network dynamics. Therefore, we will further refine the status benchmark setting methods by means of machine learning techniques in future works, especially the hyper parameter tuning [28]. Specifically, we have the plan that makes use of the deep reinforcement learning (DRL) [29] to fit the values of benchmarks. DRL combines the reinforcement learning with the deep neural network (DNN) so that it learns the optimal policy by estimating the cumulative rewards gained from the current action committed to the environment. Should the settings of benchmarks be considered as the action and the accuracy of the monitoring as the reward, DRL is able to iteratively optimize the policy on how the values of benchmarks can be set in an automatic, adaptive and online manner. The core to design DRL for benchmarks evaluation lies in the modeling of reward that reflects the monitoring accuracy, on which we will conduct a deep research in the recent future work. Concretely, deep deterministic policy gradient (DDPG) [30] is considered a feasible candidate framework to generate continuous benchmarks value estimation.

Table 2. An example of adaptive MonLink settings.

	Properties	Notations	Values	Unit
Interval Settings	Periodic Interval	$I_{periodic}$	15	s
	Mininum Interval	$I_{min}$	1	s
	Base Interval	$I_{base}$	14	s
Status Benchmark	Bandwidth	$S_{bench,1}$	20,000,000	bps
	Delay	$S_{bench,2}$	100	$\mathsf{\mu }$s
	Packet Loss	$S_{bench,3}$	0.05	N/A
	Jitter	$S_{bench,4}$	50	$\mathsf{\mu }$s

exhibits the current interval and status benchmark settings.

Algorithm 1 gives the details of how adaptive MonLink works. For each iteration, status information of every monitored equipment is collected to compute the equipment-specific MonLink interval. The smallest of all equipment-specific MonLink intervals was selected as the MonLink multicast interval to conduct next round MonLink monitoring.

Algorithm 1 Adaptive MonLink.

1: initialize $I_{min}$, $I_{base}$, $I_{periodic}$ 2: initialize j-dimensional status benchmark $S_{bench}$ 3: $I_{mc}\left(1\right)=I_{periodic}$ 4: for$(t=1;;t++)$do 5:  wait for $I_{mc}\left(t\right)$ seconds before MonLink multicast 6:  $IS(t+1)=\varnothing$ 7:  for $(i=1;i\le M;i++)$ do 8:   collect j-dim $S_i\left(t\right)$ for the i-th monitored equipment 9:   calculate next interval $I_i(t+1)$ based on Equation (1) 10:   $IS(t+1)=IS(t+1)\cup \left\{I_i(t+1)\right\}$ 11:  end for 12:  $I_{mc}(t+1)=min\left(IS(t+1)\right)$ 13: end for

To illustrate how adaptive MonLink works, an example is given in

Table 3. An example of adaptive MonLink intervals.

Round	Metrics	Value	Distance	Decay	Interval (s)
1	Bandwidth	100 Mbps	0	1	15
	Delay	10 $\mathsf{\mu }$s
	Packet Loss	0.005
	Jitter	1 $\mathsf{\mu }$s
2	Bandwidth	99 Mbps	0.055	0.947	14
	Delay	9 $\mathsf{\mu }$s
	Packet Loss	0.004
	Jitter	0.9 $\mathsf{\mu }$s
3	Bandwidth	120 Mbps	1.051	0.35	6
	Delay	11 $\mathsf{\mu }$s
	Packet Loss	0.005
	Jitter	2 $\mathsf{\mu }$s
4	Bandwidth	95 Mbps	1.25	0.286	5
	Delay	10 $\mathsf{\mu }$s
	Packet Loss	0.005
	Jitter	1 $\mathsf{\mu }$s
5	Bandwidth	100 Mbps	0.262	0.769	12
	Delay	10 $\mathsf{\mu }$s
	Packet Loss	0.005
	Jitter	5 $\mathsf{\mu }$s
6	Bandwidth	100 Mbps	0	1	15
	Delay	10 $\mathsf{\mu }$s
	Packet Loss	0.005
	Jitter	5 $\mathsf{\mu }$s
7	Bandwidth	99 Mbps	1.303	0.272	5
	Delay	7 $\mathsf{\mu }$s
	Packet Loss	0.07
	Jitter	2 $\mathsf{\mu }$s
8	Bandwidth	100 Mbps	1.301	0.272	5
	Delay	10 $\mathsf{\mu }$s
	Packet Loss	0.005
	Jitter	1 $\mathsf{\mu }$s
9	Bandwidth	100 Mbps	0.01	0.99	15
	Delay	11 $\mathsf{\mu }$s
	Packet Loss	0.005
	Jitter	1 $\mathsf{\mu }$s

. Suppose a switch had varying metrics, e.g., bandwidth, delay, packet loss, jitter, etc., where every metric had equal weight $W_j=1$ (weights can also be normalized). At iteration one, adaptive MonLink interval was the same as periodic MonLink interval (15 s), and the status information of the monitored switch is shown in the first line of Table 3. For violently changed metrics, greater decay caused by longer Euclidean distance resulted in much shorter intervals at the current iteration so as to capture system dynamics. For example, bandwidth (125 Mpbs) in iteration three was much greater that of iteration two (99 Mbps), thus a very small interval as short as 6 s. On the contrary, mildly changed metrics caused only minor interval changes (14 s) interval at iteration two compared with iteration one. Note also that at iteration six, collected status information was the same as that of iteration five, therefore, the interval increases back to the original value 15 s.

2.6. The Workflow

The overall workflow of MonLink that combines the periodic, adaptive and proactive modes are shown in Figure 3. When running MonLink, at least two threads must be launched, one for the invocation-triggered sequences (i.e., the proactive MonLink) and one for the time-triggered sequences (i.e., the periodic MonLink and adaptive MonLink), so that different modes can work together simultaneously to achieve greater flexibility. The gray sequences in Figure 3 are the core MonLink functions shared by periodic, adaptive and proactive modes, whereas blue ones, yellow ones and green ones are periodic, proactive and adaptive MonLink sequences, respectively.

2.7. The Implementation and APIs

The MonLink should be deployed at southbound interface where LLDP takes effect. That is to say both the controller and the switch must be modified to support MonLink. Specifically, the controller must be able to generate empty metric TLVs for the switches to fill in as well as to properly parse metric TLVs. Thus the code at the controller side governing LLDPDUs generation and parsing must be modified. Meanwhile, switches must be able to properly response to LLDPDUs containing metric TLVs. Therefore, corresponding code must be modified at switch side as well. Thanks to the idea that MonLink works in a piggyback fashion based on LLDP, the code modification was quite brief. We implemented MonLink using Floodlight [31] controller and Open vSwitch (OVS) [25]. In our implementation, about 20 lines of code were modified in OVS and about 60 lines of code modified in Floodlight to support basic functionalities of MonLink. Although code modification seems brief, to determine the right places for modification and re-compilation were challenging and subtle. Besides, various REST (Representational State Transfer) APIs were implemented for easier invocation of MonLink functionalities. Based on these REST APIs and backend code modification at both controller and switch sides, the MonLink platform was implemented as shown in Figure 4. Figure 4 exhibits the real-time status information (bandwidth, delay, jitter and packet loss) of a switch port. Also, upper-layer status-sensitive applications can be built on top of MonLink by invoking REST APIs ranging from basic status information collecting, optimal path planning, to more sophisticated traffic steering.

3. Results

Various experiments on MonLink were conducted in this section to test its functionalities and performance, under different software-defined energy internet topologies managed by SDN controllers: a simple ring topology, a mesh topology and a fat-tree topology, as shown in Figure 5a–c. Hosts (e.g., h1, h2, etc.) in these figures represent power grid specific nodes such as battery sensors, surveillance cameras, etc., that generated energy internet-related data forwarded by MonLink-compatible OVS switches (e.g., s1, s2, etc.) controlled by the MonLink-enabled SDN controller. The SDN controller monitored the real-time topology and status information of all MonLink-compatible switches. MonLink was compared with sFlow (see details in Section 4.2), an excellent network monitoring solution, in these experiments. All topologies were deployed in Mininet [32] with Floodlight as the controller and OVS as OpenFlow switches. The experiment configuration was: Ubuntu 14.04 server with 64 GB memory, and 40 logical 1200 MHz CPUs.

3.1. Topology Discovery

In this section, sFlow and MonLink were compared to test their capabilities of acquiring topological information. A mesh topology (Figure 5b) was deployed for this experiment. Experimental results are shown in Figure 6, the snapshots taken from sFlow and MonLink Web GUIs, respectively.

The sFlow, as introduced above, was a network monitoring system, which also provided topology discovery functionality. However, the topological information acquisition was inaccurate as shown in Figure 6a, which is quite different from the deployed mesh topology. The result indicates that sFlow’s topology discovery did not work well in SDN environments compared with the SDN-native LLDP topology discovery. As mentioned earlier, it is a desirable feature in energy internet if smart power equipments can be discovered, monitored and controlled by a smart central hub once they are installed possibly through some automated mechanics. Nevertheless, the inaccuracy in sFlow-based topology auto-discovery prevented its application in energy internet. The result also implied that if the controller wanted to acquire both topological information and status information by means of sFlow, sFlow (purely for status collecting) and LLDP (purely for topology discovery) needed to be deployed alongside and collaborate with each other. This, as a consequence, put more deployment and administrative burden on network administrators, on the one hand; on the other hand, the status information collected by sFlow and topological information collected by LLDP needed to be carefully aggregated to generate a consistent status-aware topology view identical to real topology, which is a challenging task that required plentiful programming efforts from controller/application developers. The misplacement of status information collected by sFlow on network elements/links discovered by LLDP would cause serious bad decision-making for status-sensitive applications.

MonLink, as explained above, is an extended protocol based on LLDP. It natively inherited well-performed topology discovery capability from LLDP. The experiment result indicated the topology discovered by MonLink is identical (Figure 6b) to real topology as shown in Figure 5b, plus the lightweight status information collecting (see Section 3.2).

3.2. Status Information Monitoring Performance

The sFlow and MonLink are compared in terms of runtime traffic overhead during status information collecting under different topologies (Figure 5a–c) and traffic settings (with or without upper layer video streaming traffic to mimic camera surveillance in energy internet). Three different schemes are tested in this section:

• Pure LLDP: LLDP merely conducted topology discovery without status information by means of LLDPDU multicasts and packet_in/out interactions. There packets can be considered as the benchmark overhead traffic for basic topology discovery. Therefore, LLDP was used as the benchmark to see how much more overhead the other two had to introduce in order to fulfil status information collecting in addition to basic topology discovery. This scheme used only one protocol, i.e., LLDP.
• MonLink: the extended LLDP collected status information in a piggyback fashion during LLDP-based topology discovery, thus a status-aware topology discovery. Only one extended protocol (MonLink) was used in this scheme. A modified Floodlight and OVS with support to MonLink were deployed to construct the experimental software-defined energy internet environment.
• sFlow + LLDP: LLDP conducted topology discovery while sFlow collected status information. Two protocols (sFlow and LLDP) were used in this scheme to fulfil the status-aware topology discovery in order to fairly compare with MonLink. Also, sFlow network monitoring system was deployed.

Since the default sampling interval of pure LLDP was 15 s, we set the sampling intervals of MonLink and sFlow as 15 s as well for fair comparison. The experiment results are shown in Figure 7e–f. Yellow columns exhibit the traffic overhead introduced by pure LLDP where solid-filled columns indicate the overhead ratio to the full traffic in terms of packets and pattern-filled columns indicate the overhead ratio to the full traffic in terms of bytes. Blue columns (both solid-filled and pattern-filled) indicate traffic overhead by MonLink while green columns indicate that by sFlow + LLDP. The x-axis indicates experiment time, namely 10, 20 and 30 min. Figure 7a–c are experiment results without video streaming while Figure 7d–f indicate results with video streaming from the server host, hosting a 3.2 GB mp4 file, to the client host, to mimic camera surveillance through IP-based streaming [33] in energy internets. Traffic was captured and analyzed using Wireshark.

According to the experimental results, we can see that the traffic overhead caused by MonLink was slightly greater compared with the pure LLDP benchmark. Take Figure 7f 10 min as an example (i.e., traffic overhead in fat-tree topology with 10 min video streaming), MonLink traffic overhead in bytes was 0.025% while pure LLDP benchmark overhead was 0.02%, achieving status awareness during topology discovery at the cost of very little traffic overhead. Nevertheless, traffic overhead caused by sFlow status information collecting was much greater than that of pure LLDP benchmark, almost 8.75 times as much, due to much larger packet encapsulation (up to 592 bytes). Similar analysis applies to Figure 7a–e.

To summarize, MonLink provided desirable status information monitoring performance with very low traffic overhead due to its piggyback methodology, thus a lightweight status information monitoring infrastructure for various status-aware applications.

4. Discussion

4.1. LLDP Extensions

In SDN arena, LLDP is the primary protocol used for topology discovery as mentioned in Section 1, widely adopted by various controllers such as Floodlight [31], ODL (OpenDaylight) [34], ONOS (Open Network Operating System) [35], Ryu [36], POX, NOX, etc. Therefore, extensions to LLDP are derived to achieve various purposes. Reference [37] proposed the LLDP-based latency monitoring scheme using the time-stamped LLDP packets to reduce control plane overhead and enhance measurement accuracy. Reference [38] implemented the tracing of hardware addresses in layer two bridged networks by providing self-defined TLVs. Reference [39] improved the topology discovery by LLDP, and proved that shortest paths can be built at the same time that the topology information is gathered, thus an enhanced topology discovery service for SDN. Note that some topology detection and forensics [40] techniques are also developed based on LLDP from the perspective of network security, which can also serve for the purpose of topology discovery.

These schemes share the similar thought with our scheme that implementing extra control plane functionalities based on LLDP does not lead to remarkable overhead and information can be piggybacked in self-defined fields. Our scheme differs from these works in that we derived various LLDP working modes that provide both self-adaptiveness and high interactiveness (by means of REST APIs, i.e., Representational State Transfer Application Programming Interfaces, at northbound interfaces) for upper layer applications. In addition, our work presents an architecturally better systematic perspective that the piggybacked LLDP-based information collecting and monitoring scheme can serve as a common underlying infrastructure for status-awareness by providing application-independent and platform-agnostic APIs. In this way, upper layer applications can focus on the business logics instead of implementing ad hoc information acquisition mechanisms.

4.2. Network Monitoring

Status information collecting can be done based on network monitoring, among which, sFlow, developed by InMon etc., is a typical representative. sFlow works by means of random traffic sampling. It provides traffic statistics from layer 2–4, enabling the real-time analysis on performance and trending as well as trouble shooting. sFlow consists of agents and collectors. Agents, as clients, are usually embedded in network devices such as switches, routers, etc., sending collected information encapsulated as sFlow packets (RFC 3176) [24] to the collector (i.e., the server) which generates traffic reports based on received information.

However, status information collecting based on sFlow in SDN requires the deployment of sFlow, with proper configuration and integration into SDN networks. This results in heavy administrative and runtime burden. Meanwhile, sFlow does not provide highly accurate topology discovery (see Section 3.1), thus requiring the collaboration with native LLDP topology discovery process. Besides, status information collected by sFlow and topological information collected by LLDP must be carefully aggregated to construct a consistent status-aware topological view. Last but not least, each sFlow packet contains 592 bytes by default, which would lead to remarkable traffic overhead. Our scheme overcomes problems seen in the sFlow-based scheme in that small sized metric TLVs collect status information during the topology discovery (See Section 3).

4.3. The Integration of SDN into Energy Internet

Recent years have also seen the integration of SDN into energy internet [1]/smart grid [41]. Early works originate mainly from industry (power grid enterprises). Reference [42] proposes the conceptual efficiency evaluation index system of complex power grid based on SDN and analytic hierarchy process (AHP). Similar work is proposed in reference [43]. However, these works are still at very early stage, in urgent need for theoretical depth and application soundness.

Recent efforts are also seen in academia. Realizing that inaccurate synchronization may lead to maliciously fluctuations for energy data monitoring, reference [41] proposes a software-defined on-path time synchronization (SD-OPTS) scheme for supervisory control and data sensing in smart grid to increase the flexibility, controllability and reliability of time synchronization. Our scheme differs in that we focus on status monitoring itself in energy internet while their work studied mechanisms that help with monitoring. Reference [44] proposes the wireless distribution network (WDN) as opposed to traditionally wired PDN. WDN is designed with SDN concept where SDN controller acts as the network manager controlling multiple agents deployed throughout the WDN. It proves the feasibility of SDN-governed wireless technologies in the field of smart grid. However, the QoS provisioning is not considered in this paper, which requires extensive status monitoring like the scheme we proposed in this paper. Reference [45] proposes the cooperative mechanism for energy transportation and storage in energy internet. The main purpose of the research is to well-utilize the redundant energy of a given Power Distribution Network, so that it is not wasted as treated in traditional PDN. It is modeled as an NP-hard Knapsack problem and heuristics are derived to solve this problem. To achieve this, a complementary SDN platform is constructed dedicated for decision making. Statistics for decision making are collected by means of flow table entries pushed from power routers to controllers, contradictory to the common SDN workflow that controllers push flow tables to switches/routers. Our scheme follows and extends the standard SDN workflow procedures thus exhibiting good compatibility.

5. Conclusions

Since LLDP is the mainstream method for topology discovery in SDN arena, extra control plane functionalities can hitchhike LLDP to achieve traffic efficiency without introducing newly designed dedicated protocol. In this paper, we proposed MonLink, a piggyback status information collecting scheme based on LLDP in software-defined energy internet. The extended metric TLVs enable the status information collecting during topology discovery. Several working modes—periodic, adaptive and proactive MonLink—are derived at southbound interface to achieve flexible collecting. Various REST APIs are implemented at northbound interface for easier upper-application invocations. Compared with traditional method based on sFlow, MonLink is a lightweight status information collecting framework. Meanwhile the hitchhiking on LLDP ensures that MonLink does not compromise or affect the normal workflow at southbound interface. In our future work, we are going to study the possibility of integrating MonLink with machine learning to improve the adaptive MonLink with adaptive status benchmark settings.