Communication Architecture for Grid Integration of Cyber Physical Wind Energy Systems

As we move toward increasing the grid integration of large-scale wind farms (WFs), reliable monitoring, protection, and control are needed to ensure grid stability. WFs are considered to be large and complex cyber physical systems owing to coupling between the electric power system and information and communication technologies (ICT). In this study, we proposed a framework for a cyber physical wind energy system (CPWES), which consists of four layers: a WF power system layer, data acquisition and monitoring layer, communication network layer, and application layer. We performed detailed network modeling for the WF system, including the wind turbines, meteorological mast (met-mast), and substation based on IEC 61400-25 and IEC 61850 standards. Network parameters and configuration were based on a real WF (Korean Southwest offshore project). The simulation results of the end-to-end delay were obtained for different WF applications, and they were compared with the timing requirements of the IEC 1646 standard. The proposed architecture represents a reference model for WF systems, and it can be used to enable the design of future CPWESs.


Introduction
There is a growing interest in increasing the penetration rate of renewable energies, such as wind power, solar energy, and biomass.Among them, great attention has been given to wind energy, and several large-scale wind farm (WF) projects are scheduled to be constructed in the near future.In South Korea, the cumulative installed wind power was about 610 MW and 835 MW by the end of 2014 and 2015, respectively [1].In September 2014, the Korean government announced a long-term plan for new and renewable energies, with a target of obtaining 5.0% of the total primary energy supply from renewable energies by 2020, and 11.0% by 2035.Wind power will supply and contribute about 18.2% of the total power supplied by new and renewable energies by 2035 [2].The planned WF projects include Southwest phase 1, Tamra, Hanlim, Daejeong, North-East, Shinchong, and Quiduck.The complete list of WF projects is available on the Korea wind power industry association (KWEIA) website, including site location, total capacity, manufacturer, and number of turbines [3].
As an increasing number of WFs are integrated into the power grid, communication infrastructure will play an important role in enabling the real-time operation, monitoring, and control of both wind turbines and the electric power grids to ensure grid stability [4].Communication infrastructure in WF is considered as an example of an industrial network that requires special consideration in network design owing to the following reasons [5][6][7][8][9][10][11][12][13]: (1) WFs are built in remote locations (onshore/offshore) where abundant wind resources are available.The remote sites lack established communication infrastructure or cellular coverage; (2) access to WF is difficult, especially in the case of offshore sites where the only means of access is by boat or helicopter.Furthermore, weather conditions may postpone/prevent access to WF for an unspecified period.Therefore, remote monitoring and control are highly required; (3) as advancements in the wind turbine industry enable wind turbines to migrate from shallow water to deep water with taller towers and blades, recent studies show the increase in the failure rate of large wind turbines, which impacts the availability of wind turbines and their operation and maintenance costs [14].Therefore, condition monitoring systems (CMS), structure health monitoring (SHM), and supervisory control and data acquisition unit (SCADA) systems are used for real-time monitoring; (4) the monitoring scope of large-scale WFs has been expanded to cover the operation status of wind turbines, generated power, meteorological masts (met-mast), and substations.Therefore, a considerable amount of data needs to be transferred between WFs and their control centers, which will increase the burden on the communication infrastructure.
The main components of a WF are wind turbines, electric systems, a substation, and met-masts.Communication infrastructure is used to connect WF components together using wired/wireless technologies.Owing to coupling between the physical system and the cyber communication network, a WF is considered to be a cyber physical system.Considerable research studies and investigations have been conducted to study the WF electric power system.However, information on the corresponding communication networks of the proposed models is limited.Furthermore, few research studies have been carried out to investigate the SCADA system and the underlying communication infrastructure.Moreover, it is difficult to find detailed information about how to design the communication network for a wind energy system, as the designs are the proprietary information of each manufacturer.The only available published information consists of outline descriptions of the communication infrastructure of a number of real WF projects [4,[15][16][17].The extent of the research on communication infrastructure and its role in supporting the grid integration of large-scale WFs is insufficient.
In this study, we propose a framework for cyber physical wind energy systems (CPWESs), which consists of four layers: a WF power system layer, data acquisition and monitoring layer, communication network layer, and application layer.We specified and explained the communication architecture of a wind turbine, met-mast, substation, and control center.We modeled the data transmission of the monitoring and protection systems in wind turbines, a met-mast, and a substation based on IEC 61400-25 and IEC 61850 standards.We considered an actual WF (Southwest offshore project, South Korea) as a case study to evaluate the network topology and configuration.We developed a network simulation platform for the performance evaluation of the proposed architecture.The proposed framework contributes by providing a reference architecture model that can be used for the design and implementation of future CPWESs.
The remainder of this paper is organized as follows.Section 2 presents the proposed cyber physical wind energy system.Section 3 explains WF modeling and assumptions.Section 4 discusses and analyzes the simulation results.Finally, Section 5 concludes and gives direction for future work.

Cyber Physical Wind Energy System
The CPWES is considered to be a large and complex system, as it comprises multiple heterogeneous domains that interact with each other.Figure 1 shows the proposed CPWES, which consists of four layers: a WF power system layer, data acquisition and monitoring layer, communication network layer, and application layer.

WF Power System Layer
This layer represents the WF electric power system, which is defined as a group of wind turbines connected together and tied to the utility through a system of transformers, transmission lines, and substations.A typical wind turbine consists of a wind turbine generator, step-up transformer, and a circuit breaker.The step-up transformer is used to step-up the generation voltage of each wind turbine.Wind turbines are divided into groups, and each group is connected to the collector bus through a circuit breaker.A high-voltage transformer is used to step-up the voltage to the transmission level.data sources.Sensor nodes and measurement devices are connected to different elements of the WF (inside wind turbine, at the met-mast, etc.), and their main function is to report measurements, such as temperature, speed, voltage, current, and pressure.The CMS of a wind turbine continuously collects a large volume of data signals in real time.Based on data collected from CMS, sensors, and other devices, the wind turbine controller (WTC) performs the control operation for the wind turbine (different operation modes).

Communication Network Layer
The communication infrastructure provides the connection between WF elements, and it is divided into two parts: the communication network inside the wind turbine (turbine area network, TAN) and between wind turbines, and the control center (farm area network, FAN).Inside a wind turbine are different communication protocols, such as the field bus, industrial ethernet protocols, and control area network (CAN).The architecture of the WF communication network is switchbased, consisting of ethernet switches and communication links in every wind turbine.The network configuration is based on point-to-point communication and a local area network (LAN).In order to link the WF network with a remote control center, different wide area network (WAN) technologies, either wired or wireless, could be configured, such as optical fiber cables, microwaves, and satellites.

Data Acquisition and Monitoring Layer
The data acquisition layer includes sensor nodes and measurement devices, which are the basic data sources.Sensor nodes and measurement devices are connected to different elements of the WF (inside wind turbine, at the met-mast, etc.), and their main function is to report measurements, such as temperature, speed, voltage, current, and pressure.The CMS of a wind turbine continuously collects a large volume of data signals in real time.Based on data collected from CMS, sensors, and other devices, the wind turbine controller (WTC) performs the control operation for the wind turbine (different operation modes).

Communication Network Layer
The communication infrastructure provides the connection between WF elements, and it is divided into two parts: the communication network inside the wind turbine (turbine area network, TAN) and between wind turbines, and the control center (farm area network, FAN).Inside a wind turbine are different communication protocols, such as the field bus, industrial ethernet protocols, and control area network (CAN).The architecture of the WF communication network is switch-based, consisting of ethernet switches and communication links in every wind turbine.The network configuration is based on point-to-point communication and a local area network (LAN).In order to link the WF network with a remote control center, different wide area network (WAN) technologies, either wired or wireless, could be configured, such as optical fiber cables, microwaves, and satellites.Each WF has a dedicated connection to a local control center for real-time monitoring and control.However, one control center can remotely manage and control one or more WFs.

Application Layer
The SCADA systems are used for data acquisition, remote monitoring, real-time control, and data recording.There are multiple applications covered by the SCADA systems in a WF.The three main applications are as follows: the turbine SCADA system provides the connectivity among wind turbines and enables remote monitoring and control for each wind turbine and their associated sub-systems, the WF SCADA system connects all devices from all wind turbines, as well as the electric substation together, and the security SCADA system provides the IP telephony services and video surveillance.The SCADA system remotely collects the process information from WF components and stores it in historical databases and servers (historical servers, metering server, meteorological server, etc.).Based on the collected information, the control center executes appropriate actions.

CPWES Modeling and Assumption
In this study, the Southwest offshore wind farm located in South Korea was considered as a case study.We aimed to design the communication infrastructure for phase 1 of the project, which consists of 20 wind turbines with a total capacity of 60 MW, as shown in Figure 2. The output voltage of each turbine was 690 V.The voltage is stepped-up to the collector bus with a typical voltage of 34.5 kV.All turbines were connected to an offshore substation.The spacing between turbines was about 800 m (along the rows and between the rows).The longest cable length was about 1.2 km between turbine No. 3 and the offshore platform, while the shortest cable length was about 524 m between turbine No. 1 and the offshore platform [18].The electric topology configuration consisted of three feeders, one bus, and one HV transformer.We assumed that the communication network topology followed the WF electric topology, where the optical fiber cables are integrated with the submarine cables.The WF communication architecture is shown in Figure 3.Each WF has a dedicated connection to a local control center for real-time monitoring and control.However, one control center can remotely manage and control one or more WFs.

Application Layer
The SCADA systems are used for data acquisition, remote monitoring, real-time control, and data recording.There are multiple applications covered by the SCADA systems in a WF.The three main applications are as follows: the turbine SCADA system provides the connectivity among wind turbines and enables remote monitoring and control for each wind turbine and their associated subsystems, the WF SCADA system connects all devices from all wind turbines, as well as the electric substation together, and the security SCADA system provides the IP telephony services and video surveillance.The SCADA system remotely collects the process information from WF components and stores it in historical databases and servers (historical servers, metering server, meteorological server, etc.).Based on the collected information, the control center executes appropriate actions.

CPWES Modeling and Assumption
In this study, the Southwest offshore wind farm located in South Korea was considered as a case study.We aimed to design the communication infrastructure for phase 1 of the project, which consists of 20 wind turbines with a total capacity of 60 MW, as shown in Figure 2. The output voltage of each turbine was 690 V.The voltage is stepped-up to the collector bus with a typical voltage of 34.5 kV.All turbines were connected to an offshore substation.The spacing between turbines was about 800 m (along the rows and between the rows).The longest cable length was about 1.2 km between turbine No. 3 and the offshore platform, while the shortest cable length was about 524 m between turbine no. 1 and the offshore platform [18].The electric topology configuration consisted of three feeders, one bus, and one HV transformer.We assumed that the communication network topology followed the WF electric topology, where the optical fiber cables are integrated with the submarine cables.The WF communication architecture is shown in Figure 3.

Traffic Model for Wind Turbine based on IEC 61400-25
In order to model the wind turbine subsystems, we considered the IEC 61400-25 standard, which is an adaption of IEC 61850 [19].IEC 61400-25 provides information for the monitoring and control of wind power plants.We assumed that a wind turbine consists of 10 logical nodes (LNs): WROT, WTEM, WGEN, WCNV, WNAC, WYAW, WTOW, WTRF, WMET and WFOU, as shown in Equation (1).Each LN represents a wind turbine sub-system, as given in Table 1 Different types of sensor nodes and measurement devices were connected to different turbine parts to measure different parameters, such as voltage (V), current (A), wind speed (WdSpd), wind direction (WdDir), temperature (Tmp), Humidity (Hum), displacement (Disp), and pressure (Pres).Equation (2) defines types of sensor nodes (SNTYPE) inside a wind turbine sub-system.
Regarding the data acquisition and monitoring layer, the authors in Ref. [20] designed a data acquisition system to be installed on a real wind turbine.The total number of sensor nodes was about 29.The sensor nodes were divided into two groups with sampling rates: 50 Hz (low speed) and 20 kHz (high speed).In our model, the total number of sensor nodes defined inside a wind turbine was 108 [21].Figure 4 shows a schematic diagram of the cyber physical model for a wind turbine system

Traffic Model for Wind Turbine Based on IEC 61400-25
In order to model the wind turbine subsystems, we considered the IEC 61400-25 standard, which is an adaption of IEC 61850 [19].IEC 61400-25 provides information for the monitoring and control of wind power plants.We assumed that a wind turbine consists of 10 logical nodes (LNs): WROT, WTEM, WGEN, WCNV, WNAC, WYAW, WTOW, WTRF, WMET and WFOU, as shown in Equation (1).Each LN represents a wind turbine sub-system, as given in Table 1.
Regarding the data acquisition and monitoring layer, the authors in Ref. [20] designed a data acquisition system to be installed on a real wind turbine.The total number of sensor nodes was about 29.The sensor nodes were divided into two groups with sampling rates: 50 Hz (low speed) and 20 kHz (high speed).In our model, the total number of sensor nodes defined inside a wind turbine was 108 [21].Figure 4 shows a schematic diagram of the cyber physical model for a wind turbine system where seven LNs are located at the turbine nacelle and three LNs are in the bottom of the tower.Two data collection units were considered for aggregating the traffic from different turbine subsystems.where seven LNs are located at the turbine nacelle and three LNs are in the bottom of the tower.Two data collection units were considered for aggregating the traffic from different turbine subsystems.
Table 1.Wind turbine logical nodes [18].To determine the amount of data generated from each sensor node (data rate, SNDR), we defined the sample size (NB), the sampling rate (FS), and the number of channels (NC), as shown in Equation ( 4).

LN
Given the displacement sensor as an example, the sensor generates 10 samples/s and the number of channels required for the measuring of data is 2 channels.The total amount of data is 320 bits/s, considering a sample size of 16 bits.Table 2 shows how to calculate the measuring requirement for different sensor nodes.We calculated the sensing data for all sensor nodes inside a wind turbine (108 sensor nodes), and Table 3 shows the traffic configuration at each wind turbine subsystem.To determine the amount of data generated from each sensor node (data rate, SN DR ), we defined the sample size (N B ), the sampling rate (F S ), and the number of channels (N C ), as shown in Equation (4).
Appl.Sci.2017, 7, 1034 7 of 20 Given the displacement sensor as an example, the sensor generates 10 samples/s and the number of channels required for the measuring of data is 2 channels.The total amount of data is 320 bits/s, considering a sample size of 16 bits.Table 2 shows how to calculate the measuring requirement for different sensor nodes.We calculated the sensing data for all sensor nodes inside a wind turbine (108 sensor nodes), and Table 3 shows the traffic configuration at each wind turbine subsystem.In this work, a WF is divided into different protection zones: the wind turbine zone, collector feeder zone, collector bus zone, and high-voltage transformer zone.Each zone has one or more protection devices associated with it, such as a wind turbine protection relay, step-up transformer protection relay, feeder protection relay, bus protection relay, and distance relay for the transmission line [22].We considered three types of IEDs: a breaker IED, merging unit (MU) IED, and protection and control (P&C) IED [23][24][25][26].Based on IEC 61850, the function of a MU IED is to acquire the voltage and current signals from the field CT and PT.The function of a CB IED is to monitor the state and control the condition of the circuit breaker.Furthermore, it receives control signals from the P&C IEDs and updates the status change to the protection IEDs.The P&C IED is a universal device that integrates protection and control functionalities and operates at the bay level in the substation.
All acquired signals from different zones were transmitted to the central protection and control (CPC) system on the offshore substation using point-to-point fiber communication.We considered one CPC system, which is responsible for all protection and control devices in the WF, as shown in Figure 5.In addition, each wind turbine generator (WTG) was configured with a merging unit to acquire current, voltage, and breaker signals for wind turbine protection.Data packets were transmitted over a communication network to a central relaying unit (CRU) for P&C functions of the whole WF.For MU-IEDs, we assumed that the sampling frequency of the voltage and current data was 6400 Hz for a 50 Hz power system, and each piece of sampling data was represented by 2 bytes [27].Considering the 3-phase voltage and current measurement, the MU-IEDs send updated values of 76,800 bytes/s to the P&C server at the local control center [25].Moreover, the CB-IED sends a status value of 16 bytes/s to the P&C server.The details of the process level, bay level, and station level are as follows:

•
Process Level: The primary equipment includes merging units (MUs), actuators, and I/O devices.

•
Bay Level: The secondary equipment includes different IEDs and protection relays.

•
Station Level: It is located on the offshore platform and includes a central protection and control unit, remote terminal units (RTU), and a human machine interface (HMI).
architecture.At the station level, there were the following: a station PC, a protection and control server, ethernet switches, and communication links.

Traffic Model for Meteorological Mast
We considered a met-mast to measure the wind condition in the WF area.Based on Reference [28], we defined the number, types, and locations of the senor nodes and measuring instruments installed at the met-mast, as shown in Figure 6.The platform height was 10 m with respect to the mean sea level, and the top of the met-mast was about 97 m.Anemometers were installed at 97 m, 96 m, 86 m, 86 m, 76 m, 66 m, 56 m, 46 m and 26 m.Wind vanes were installed at 96 m, 76 m, 56 m and 46 m.Barometers and temperature and humidity sensors were installed at 94 m and 13 m.The details of the sensor types and their locations at the met-mast are shown in Table 6.The acoustic doppler current profile (ADCP) is used to measure the current direction and velocity at sea level.The communication network was constructed based on ethernet-based architecture.The offshore substation consisted of three feeders, one bus, and one transformer.Protection devices at wind turbines, feeders, and bus were modeled as a subnet consisting of one CB-IED, one MU-IED, one P&C-IED, and one ethernet switch.The step-up transformer was modeled as a subnet consisting of one CB-IED, one MU-IED, two P&C IED, and an ethernet switch.Table 4 shows the IEDs configuration.The traffic configuration and data flow for the CB-IED and MU-IED considered in our model are given in Table 5. IEDs and relays were connected through a 100 Mbps ethernet-based architecture.At the station level, there were the following: a station PC, a protection and control server, ethernet switches, and communication links.

Traffic Model for Meteorological Mast
We considered a met-mast to measure the wind condition in the WF area.Based on Reference [28], we defined the number, types, and locations of the senor nodes and measuring instruments installed at the met-mast, as shown in Figure 6.The platform height was 10 m with respect to the mean sea level, and the top of the met-mast was about 97 m.Anemometers were installed at 97 m, 96 m, 86 m, 86 m, 76 m, 66 m, 56 m, 46 m and 26 m.Wind vanes were installed at 96 m, 76 m, 56 m and 46 m.Barometers and temperature and humidity sensors were installed at 94 m and 13 m.The details of the sensor types and their locations at the met-mast are shown in Table 6.The acoustic doppler current profile (ADCP) is used to measure the current direction and velocity at sea level.

Wind Farm Timing Requirements
In a power system, it is important to meet the latency requirement for different real-time monitoring and protection applications.Therefore, the WF communication network should ensure the end-to-end delay for data transmission between wind turbines, the met-mast, the substation, and the control center.The IEC 61400-25 standard does not provide any specific timing requirements for the WF.In this study, we considered the latency requirement based on the electric power system.Table 7 shows the communication timing requirements for electric substation automation based on IEEE 1646 [29].We mapped WF monitoring and control information as follows: protection  the end-to-end delay for data transmission between wind turbines, the met-mast, the substation, and the control center.The IEC 61400-25 standard does not provide any specific timing requirements for the WF.In this study, we considered the latency requirement based on the electric power system.Table 7 shows the communication timing requirements for electric substation automation based on IEEE 1646 [29].We mapped WF monitoring and control information as follows: protection information from IEDs were mapped for protection information, analogue measurement and status information were mapped for monitoring and control information, and meteorological data from the met-mast were mapped for operation and maintenance.

Simulation Results
The WF communication architecture is modeled using the OPNET Modeler [30], as shown in Figure 3.The proposed communication architecture for the Southwest WF consists of 22 subnetworks (20 wind turbines, one met-mast, and one offshore substation) [31].Upstream data from WF subsystems are transmitted to the local control center at the offshore substation.We simulated different scenarios for WF monitoring, protection, and control, as shown in Table 8.The details of simulation cases are as follows:

•
Standalone wind turbine: includes data transmission of the condition monitoring system during different operation modes.

•
Wind farm: consists of 20 wind turbines configured in a cascade architecture.

•
Met-mast: includes data transmission of sensors and measurement devices installed at the met-mast.

•
Substation: consists of protection and control devices (IEDs) at wind turbines, feeders, and offshore substation.

Standalone Wind Turbine Results
We considered four operation modes for a wind turbine, which are idle, start-up, power generation, and shutdown, as shown in Figure 7. Based on wind conditions, the WTC determines the turbine operation mode [8].We assumed that the control center operator turned on a wind turbine for operation.The WTC first checked the condition of the wind turbine.If no faults were detected in the wind turbine system, the WTC activated the operation in the idle mode.During the idle mode, if the average wind speed was greater than the cut-in wind speed for a certain time, the WTC activated the start-up operation mode.During the power production mode, the WTC adjusted the blades' pitch angle and position of the nacelle in order to optimize the turbine operation.In case of higher wind speeds, which were greater than the cut-off, the WTC activated the shutdown mode.

Data Type Idle Mode Operation Mode Shutdown Mode
The OPNET Modeler was used to simulate data transmission of condition monitoring system for a wind turbine during three operation modes (shutdown, idle, and operation).The nacelle dimension was configured as 12 m × 4 m.The total number of monitoring parameters and sensors was 108.The height of the tower was configured as 85 m (distance between DCU and main ethernet switch).Based on the IEC 61400-25 standard, different profiles were defined, configured, and assigned to each LN.The simulation time was 20 min.
Figure 8a shows the total received traffic at the WTC for different operation modes.The maximum received traffic was about 227,036 bytes/s (225,544 AM, 58 SI, and 1434 FOU) during turbine operation, while the received traffic was about 5808 bytes/s during the idle mode.The difference was due to the sensing data of the electric measurement (voltage, current, power, power factor and frequency), which were about 221,228 bytes/s.No traffic was received at the WTC while the turbine was in the shutdown mode.We classified the sensor nodes inside the wind turbine into four different categories: mechanical measurement (rotor speed, pitch angle, displacement, vibration, etc.), electrical measurements (voltage, current, power, frequency, etc.), meteorological data (wind speed, wind direction, temperature, humidity, etc.), and foundation measurements, as shown in Table 9.Based on the turbine operation mode, the amount of monitoring data forwarded to the WTC was different.This classification meant that some of the sensor nodes were continuously working, while other sensor nodes were generating the monitoring data based on the turbine operation mode.

Data Type Idle Mode Operation Mode Shutdown Mode
The OPNET Modeler was used to simulate data transmission of condition monitoring system for a wind turbine during three operation modes (shutdown, idle, and operation).The nacelle dimension was configured as 12 m × 4 m.The total number of monitoring parameters and sensors was 108.The height of the tower was configured as 85 m (distance between DCU and main ethernet switch).Based on the IEC 61400-25 standard, different profiles were defined, configured, and assigned to each LN.The simulation time was 20 min.
Figure 8a shows the total received traffic at the WTC for different operation modes.The maximum received traffic was about 227,036 bytes/s (225,544 AM, 58 SI, and 1434 FOU) during turbine operation, while the received traffic was about 5808 bytes/s during the idle mode.The difference was due to the sensing data of the electric measurement (voltage, current, power, power factor and frequency), which were about 221,228 bytes/s.No traffic was received at the WTC while the turbine was in the shutdown mode.
Figure 8b shows the end-to-end delay inside the wind turbine between sensor nodes and the WTC).The link capacities inside the wind turbine were configured with 10 Mbps and 100 Mbps.During the power production mode, the end-to-end delay was about 0.098 ms and 1.1 ms using link capacities of 10 Mbps and 100 Mbps, respectively.The received traffic from the pitch angel sensor, wind speed sensor, and tilt sensor were 18 bytes/s, 36 bytes/s, and 40 bytes/s, respectively, as shown in Figure 9a.The measurements were received at the WTC just after the control center operator activated the wind turbine for operation at t = 300 s.For the electric measurement, the received traffic for the current and voltage were 73,728 bytes/s and 147,456 bytes/s, respectively.Note that the electric measurements were transmitted to the WTC at t = 420 s, and no data was received during the idle mode, as shown in Figure 9b.
The OPNET Modeler was used to simulate data transmission of condition monitoring system for a wind turbine during three operation modes (shutdown, idle, and operation).The nacelle dimension was configured as 12 m × 4 m.The total number of monitoring parameters and sensors was 108.The height of the tower was configured as 85 m (distance between DCU and main ethernet switch).Based on the IEC 61400-25 standard, different profiles were defined, configured, and assigned to each LN.The simulation time was 20 min.
Figure 8a shows the total received traffic at the WTC for different operation modes.The maximum received traffic was about 227,036 bytes/s (225,544 AM, 58 SI, and 1434 FOU) during turbine operation, while the received traffic was about 5808 bytes/s during the idle mode.The difference was due to the sensing data of the electric measurement (voltage, current, power, power factor and frequency), which were about 221,228 bytes/s.No traffic was received at the WTC while the turbine was in the shutdown mode.Figure 8b shows the end-to-end delay inside the wind turbine between sensor nodes and the WTC).The link capacities inside the wind turbine were configured with 10 Mbps and 100 Mbps.During the power production mode, the end-to-end delay was about 0.098 ms and 1.1 ms using link capacities of 10 Mbps and 100 Mbps, respectively.The received traffic from the pitch angel sensor, wind speed sensor, and tilt sensor were 18 bytes/s, 36 bytes/s, and 40 bytes/s, respectively, as shown in Figure 9a.The measurements were received at the WTC just after the control center operator activated the wind turbine for operation at t = 300 s.For the electric measurement, the received traffic for the current and voltage were 73,728 bytes/s and 147,456 bytes/s, respectively.Note that the electric measurements were transmitted to the WTC at t = 420 s, and no data was received during the idle mode, as shown in Figure 9b.In the future, wind turbines will integrate more and more sensor nodes and monitoring devices.This will require the communication network to support even higher bandwidth.We studied the end-to-end delay for the case where the number of sensor nodes inside the wind turbine is duplicated two-times (case2×) and three-times (case3×).The results in Figure 10 shows the received traffic and the end-to-end delay at the WTC during different operation modes.The end-to-end delay increased from 1.1 ms (base case with 108 sensor nodes) to 3.08 ms for case3×.The results of the ETE delay were proportional to the network traffic and number of sensor nodes inside the wind turbine, as shown in Table 10.In the future, wind turbines will integrate more and more sensor nodes and monitoring devices.This will require the communication network to support even higher bandwidth.We studied the end-to-end delay for the case where the number of sensor nodes inside the wind turbine is duplicated two-times (case2×) and three-times (case3×).The results in Figure 10 shows the received traffic and the end-to-end delay at the WTC during different operation modes.The end-to-end delay increased from 1.1 ms (base case with 108 sensor nodes) to 3.08 ms for case3×.The results of the ETE delay were proportional to the network traffic and number of sensor nodes inside the wind turbine, as shown in Table 10.
end-to-end delay for the case where the number of sensor nodes inside the wind turbine is duplicated two-times (case2×) and three-times (case3×).The results in Figure 10 shows the received traffic and the end-to-end delay at the WTC during different operation modes.The end-to-end delay increased from 1.1 ms (base case with 108 sensor nodes) to 3.08 ms for case3×.The results of the ETE delay were proportional to the network traffic and number of sensor nodes inside the wind turbine, as shown in Table 10.

Wind Farm Results
We configured the WF communication network based on the data from real large-scale WF projects (Greater Gabbard WF in UK [16], Horn Rev WF in Denmark [15], and Yeung Heung in South Korea [17]).The network topology was configured as cascaded architecture.A main ethernet switch was located on the offshore platform.Dedicated optical fibers were connected between the main ethernet switch and the nearest wind turbines.The connectivity between turbines was provided through the connection between ethernet switches installed at the base of each turbine.
Figure 11 shows the total traffic received at the control center (SCADA server).The traffic received for analogue measurements, status information, foundation measurements, and protection information was 4,510,880 bytes/s (225,544 × 20), 1160 bytes/s (58 × 20), 28,680 bytes/s (1434 × 20), and 1,536,320 bytes/s (76,816 × 20), respectively.Figure 12 shows the average ETE delay of real-time monitoring data for all wind turbine applications.The average ETE delay was about 7.83 ms and 0.78 ms for link capacities of 100 Mbps and 1 Gbps, respectively.For the 20 wind turbines in the WF, Table 11 shows the maximum and minimum ETE delays for SCADA, MU-IED, and CB-ID between the wind turbines and the control center.12 shows the average ETE delay of real-time monitoring data for all wind turbine applications.The average ETE delay was about 7.83 ms and 0.78 ms for link capacities of 100 Mbps and 1 Gbps, respectively.For the 20 wind turbines in the WF, Table 11 shows the maximum and minimum ETE delays for SCADA, MU-IED, and CB-ID between the wind turbines and the control center.information was 4,510,880 bytes/s (225,544 × 20), 1160 bytes/s (58 × 20), 28,680 bytes/s (1434 × 20), and 1,536,320 bytes/s (76,816 × 20), respectively.Figure 12 shows the average ETE delay of real-time monitoring data for all wind turbine applications.The average ETE delay was about 7.83 ms and 0.78 ms for link capacities of 100 Mbps and 1 Gbps, respectively.For the 20 wind turbines in the WF, Table 11 shows the maximum and minimum ETE delays for SCADA, MU-IED, and CB-ID between the wind turbines and the control center.

Met-Mast Results
We assumed that the dimension of the Met-mast platform were 10 × 8 m.Four data collection units (DCU) were installed at different levels (90 m, 70 m, 50 m, and 20 m) for collecting the measurement data.All sensing data were collected at the measurement PC located at the met-mast platform.Each DCU had a dedicated communication link to the measurement PC to maintain reception of data in case of any faults in other DCUs.The electric power needed for the measurement equipment and other devices at the met-mast could be supported from the WF or from photovoltaic panels.Figure 13 shows the received traffic at the measurement PC for pressure, ADCP, humidity, wind speed, and wind direction, which were 150 bytes/s, 400 bytes, 4 bytes/s, 48 bytes, and 24 bytes/s, respectively.Figure 14a shows the total received data at the measurement PC of about 638 bytes/s, while the ETE delay is about 0.54 ms and 0.058 ms for link capacities of 10 Mbps and 100 Mbps, respectively.The amount of data stored at the measurement PC was about 55 Mb per day, and data was transmitted to the met-mast server at the local control center for storage.
In the real system, the data acquisition system at the met-mast uses wireless data transmission using WCDMA communication to transmit the collected data from the measurement PC to the remote control center [32].We assumed that the met-mast was connected to the nearest wind turbine in the WF (WT16).The real-time measured data were stored at the measurement PC and transmitted to the local control center (met-mast server) through the WF communication network.The IP ETE delay of the met-mast data through the WF network is shown in Figure 15.

Substation Automation Results
The communication network for the offshore WF substation is shown in Figure 3.The architecture is based on IEC 61850 with three levels: a process level, bay level, and station level [33].We assumed that all MU IEDs and CB IEDs were sending metering values (76,800 bytes/s) and the status of the breaker (16 bytes/s) to the central P&C unit at the station level on the offshore platform.Three scenarios were configured with different link capacities of 10, 100 and 1000 Mbps.To validate the results, we compared the generated amount of traffic from different IEDs with the received traffic from the server.The server FTP traffic received was about 416 bytes/s (26 CB-IEDs × 16 bytes/s) and 1,920,000 (25 MU-IEDs × 76,800 bytes/s) for the CB-IEDs and MU-IEDs, respectively.Figure 16 shows the traffic received, which agrees with our calculations.In the real system, the data acquisition system at the met-mast uses wireless data transmission using WCDMA communication to transmit the collected data from the measurement PC to the remote control center [32].We assumed that the met-mast was connected to the nearest wind turbine in the WF (WT16).The real-time measured data were stored at the measurement PC and transmitted to the local control center (met-mast server) through the WF communication network.The IP ETE delay of the met-mast data through the WF network is shown in Figure 15.

Substation Automation Results
The communication network for the offshore WF substation is shown in Figure 3.The architecture is based on IEC 61850 with three levels: a process level, bay level, and station level [33].We assumed that all MU IEDs and CB IEDs were sending metering values (76,800 bytes/s) and the status of the breaker (16 bytes/s) to the central P&C unit at the station level on the offshore platform.Three scenarios were configured with different link capacities of 10, 100 and 1000 Mbps.To validate

Substation Automation Results
The communication network for the offshore WF substation is shown in Figure 3.The architecture is based on IEC 61850 with three levels: a process level, bay level, and station level [33].We assumed that all MU IEDs and CB IEDs were sending metering values (76,800 bytes/s) and the status of the breaker (16 bytes/s) to the central P&C unit at the station level on the offshore platform.Three scenarios were configured with different link capacities of 10, 100 and 1000 Mbps.To validate the results, we compared the generated amount of traffic from different IEDs with the received traffic from the server.The server FTP traffic received was about 416 bytes/s (26 CB-IEDs × 16 bytes/s) and 1,920,000 (25 MU-IEDs × 76,800 bytes/s) for the CB-IEDs and MU-IEDs, respectively.Figure 16 shows the traffic received, which agrees with our calculations.

Substation Automation Results
The communication network for the offshore WF substation is shown in Figure 3.The architecture is based on IEC 61850 with three levels: a process level, bay level, and station level [33].We assumed that all MU IEDs and CB IEDs were sending metering values (76,800 bytes/s) and the status of the breaker (16 bytes/s) to the central P&C unit at the station level on the offshore platform.Three scenarios were configured with different link capacities of 10, 100 and 1000 Mbps.To validate the results, we compared the generated amount of traffic from different IEDs with the received traffic from the server.The server FTP traffic received was about 416 bytes/s (26 CB-IEDs × 16 bytes/s) and 1,920,000 (25 MU-IEDs × 76,800 bytes/s) for the CB-IEDs and MU-IEDs, respectively.Figure 16 shows the traffic received, which agrees with our calculations.Figure 17 shows the average ETE delay for all protection and control information considering link capacities of 100 Mbps and 1 Gbps.Compared with the timing requirement of the IEEE 1646 standard, the link capacity of 100 Mbps cannot satisfy the delay requirement owing to a high delay of about 8.5 ms.The link capacity of 1 Gbps can satisfy the protection requirement with a delay less than 1 ms, as shown in Figure 17. Figure 17 shows the average ETE delay for all protection and control information considering link capacities of 100 Mbps and 1 Gbps.Compared with the timing requirement of the IEEE 1646 standard, the link capacity of 100 Mbps cannot satisfy the delay requirement owing to a high delay of about 8.5 ms.The link capacity of 1 Gbps can satisfy the protection requirement with a delay less than 1 ms, as shown in Figure 17.It was observed that using 10 Mbps for channel capacity was not sufficient, as the amount of generated traffic was much higher than the link capacity.Figures 18 and 19 show the worst case scenario in case of 10 Mbps where data loss and higher delays occur.Table 12 shows the ETE delay of the MU-IEDs and CB-IEDs for the feeders (F1, F2, and F3), bus, and transformer with link bandwidth of 100 Mbps and 1 Gbps.Figure 20 shows the ETE delay of the MU-IED and CB-IED at It was observed that using 10 Mbps for channel capacity was not sufficient, as the amount of generated traffic was much higher than the link capacity.Figures 18 and 19 show the worst case scenario in case of 10 Mbps where data loss and higher delays occur.Table 12 shows the ETE delay of the MU-IEDs and CB-IEDs for the feeders (F1, F2, and F3), bus, and transformer with link bandwidth of 100 Mbps and 1 Gbps.Figure 20 shows the ETE delay of the MU-IED and CB-IED at the collector bus.It was observed that using 10 Mbps for channel capacity was not sufficient, as the amount of generated traffic was much higher than the link capacity.Figures 18 and 19 show the worst case scenario in case of 10 Mbps where data loss and higher delays occur.Table 12 shows the ETE delay of the MU-IEDs and CB-IEDs for the feeders (F1, F2, and F3), bus, and transformer with link bandwidth of 100 Mbps and 1 Gbps.Figure 20 shows the ETE delay of the MU-IED and CB-IED at the collector bus.It was observed that using 10 Mbps for channel capacity was not sufficient, as the amount of generated traffic was much higher than the link capacity.Figures 18 and 19 show the worst case scenario in case of 10 Mbps where data loss and higher delays occur.Table 12 shows the ETE delay of the MU-IEDs and CB-IEDs for the feeders (F1, F2, and F3), bus, and transformer with link bandwidth of 100 Mbps and 1 Gbps.Figure 20 shows the ETE delay of the MU-IED and CB-IED at the collector bus.

Conclusions
In this work, we proposed a framework for the grid integration of a cyber physical wind energy system (CPWES) that consists of four layers: a wind farm power system layer, data acquisition and monitoring layer, communication network layer, and application layer.A real WF project (Korean Southwest -phase 1) was considered as a case study.We developed a communication network model for the WF based on the OPNET Modeler.The network architecture was modeled based on IEC 61400-25 and IEC 61850 standards.Different scenarios were configured for evaluating the communication network performance at different levels, including: data transmission of condition monitoring system for a standalone wind turbine during different operation modes, communication network for a real WF consists of 20 wind turbines, data transmission of sensors and measurement devices installed at the met-mast, and communication network for protection and control devices at wind turbines, feeders, and offshore substation.The proposed architecture was evaluated with respect to network topology, link capacity, and end-to-end delay for different applications.Simulation results indicated the channel capacity of 1 Gbps satisfied the requirement of the IEEE 1646 standard for the WF system.This work contributes by providing a reference architecture for modeling, simulating, and evaluating the communication network of a WF and can be used for the design and implementation of future CPWESs.

Conclusions
In this work, we proposed a framework for the grid integration of a cyber physical wind energy system (CPWES) that consists of four layers: a wind farm power system layer, data acquisition and monitoring layer, communication network layer, and application layer.A real WF project (Korean Southwest -phase 1) was considered as a case study.We developed a communication network model for the WF based on the OPNET Modeler.The network architecture was modeled based on IEC 61400-25 and IEC 61850 standards.Different scenarios were configured for evaluating the communication network performance at different levels, including: data transmission of condition monitoring system for a standalone wind turbine during different operation modes, communication network for a real WF consists of 20 wind turbines, data transmission of sensors and measurement devices installed at the met-mast, and communication network for protection and control devices at wind turbines, feeders, and offshore substation.The proposed architecture was evaluated with respect to network topology, link capacity, and end-to-end delay for different applications.Simulation results indicated the channel capacity of 1 Gbps satisfied the requirement of the IEEE 1646 standard for the WF system.This work contributes by providing a reference architecture for modeling, simulating, and evaluating the communication network of a WF and can be used for the design and implementation of future CPWESs.

Figure 1 .
Figure 1.Proposed architecture of the cyber physical wind energy system.SCADA: supervisory control and data acquisition; CCTV: closed-circuit television; VOIP: voice over internet protocol; IED: intelligent electronic device; WTG: wind turbine generator; WTC: wind turbine controller; HV: high voltage; Met.Mast: meteorological mast.

Figure 1 .
Figure 1.Proposed architecture of the cyber physical wind energy system.SCADA: supervisory control and data acquisition; CCTV: closed-circuit television; VOIP: voice over internet protocol; IED: intelligent electronic device; WTG: wind turbine generator; WTC: wind turbine controller; HV: high voltage; Met.Mast: meteorological mast.

Figure 2 .
Figure 2. Configuration of the Southwest wind farm project (Test WF).Figure 2. Configuration of the Southwest wind farm project (Test WF).

Figure 2 .
Figure 2. Configuration of the Southwest wind farm project (Test WF).Figure 2. Configuration of the Southwest wind farm project (Test WF).

Figure 3 .
Figure 3. Proposed OPNET model for the wind farm communication architecture.OPNET: optimized network engineering tool.

Figure 3 .
Figure 3. Proposed OPNET model for the wind farm communication architecture.OPNET: optimized network engineering tool.
information WNAC Wind turbine nacelle information WTRM Wind turbine transmission information WYAW Wind turbine yawing information WTGEN Wind turbine generator information WTOW Wind turbine tower information WCNV Wind turbine converter information WFOU Wind turbine foundation information WTRF Wind turbine transformer information WMET Wind power plant meteorological information

Figure 4 .
Figure 4. Schematic diagram of the cyber physical model for the wind turbine system.

Figure 4 .
Figure 4. Schematic diagram of the cyber physical model for the wind turbine system.

Figure 5 .
Figure 5. Schematic diagram of the cyber physical model for the wind farm substation.

Figure 5 .
Figure 5. Schematic diagram of the cyber physical model for the wind farm substation.

Figure 6 .
Figure 6.Schematic diagram of the cyber physical model for met-mast.

Figure 6 .
Figure 6.Schematic diagram of the cyber physical model for met-mast.

Figure 7 .
Figure 7. Power curve and general operation modes of a typical wind turbine.

Figure 8 .
Figure 8.(a) Total received traffic at WTC for different operation modes; (b) End-to-end delay for the internal communication inside the wind turbine.

Figure 7 .
Figure 7. Power curve and general operation modes of a typical wind turbine.

Figure 8 .
Figure 8.(a) Total received traffic at WTC for different operation modes; (b) End-to-end delay for the internal communication inside the wind turbine.Figure 8. (a) Total received traffic at WTC for different operation modes; (b) End-to-end delay for the internal communication inside the wind turbine.

Figure 8 .
Figure 8.(a) Total received traffic at WTC for different operation modes; (b) End-to-end delay for the internal communication inside the wind turbine.Figure 8. (a) Total received traffic at WTC for different operation modes; (b) End-to-end delay for the internal communication inside the wind turbine.

Figure 9 .
Figure 9. Traffic received at WTC for different sensors.(a) Wind speed, pitch angle, tilt; (b) Voltage and current.

Figure 9 .
Figure 9. Traffic received at WTC for different sensors.(a) Wind speed, pitch angle, tilt; (b) Voltage and current.

Figure 10 .
Figure 10.(a) Total received traffic at WTC for different operation mode; (b) ETE delay for the internal communication inside wind turbine.Figure 10.(a) Total received traffic at WTC for different operation mode; (b) ETE delay for the internal communication inside wind turbine.

Figure 10 .
Figure 10.(a) Total received traffic at WTC for different operation mode; (b) ETE delay for the internal communication inside wind turbine.Figure 10.(a) Total received traffic at WTC for different operation mode; (b) ETE delay for the internal communication inside wind turbine.

Figure 11 .
Figure 11.Total received traffic at the SCADA server in the control center (20 wind turbines).

Figure 12 .
Figure 12.Average ETE delay for the wind farm data (20 wind turbines).

Figure 11 .
Figure 11.Total received traffic at the SCADA server in the control center (20 wind turbines).

Figure 11 .
Figure 11.Total received traffic at the SCADA server in the control center (20 wind turbines).

Figure 12 .
Figure 12.Average ETE delay for the wind farm data (20 wind turbines).Figure 12. Average ETE delay for the wind farm data (20 wind turbines).

Figure 12 .
Figure 12.Average ETE delay for the wind farm data (20 wind turbines).Figure 12. Average ETE delay for the wind farm data (20 wind turbines).

Figure 13 .
Figure 13.Traffic received at measurement PC for different sensors.(a) Voltage and current; (b) Wind speed, pitch angle, and tilt.

Figure 14 .
Figure 14.(a) Total received traffic at measurement PC for different sensors (b) ETE delay for the met-mast sensors.

Figure 14 .
Figure 14.(a) Total received traffic at measurement PC for different sensors (b) ETE delay for the metmast sensors.

Figure 19 .
Figure 19.ETE delay for P&C information in the WF (channel capacity 10 Mbps).

Figure 19 .
Figure 19.ETE delay for P&C information in the WF (channel capacity 10 Mbps).Figure 19.ETE delay for P&C information in the WF (channel capacity 10 Mbps).

Figure 19 .
Figure 19.ETE delay for P&C information in the WF (channel capacity 10 Mbps).Figure 19.ETE delay for P&C information in the WF (channel capacity 10 Mbps).

Figure 20 .
Figure 20.ETE delay for the MU-IED and CB-IED at the collector bus (Link capacity 100 Mbps, 1 Gbps).

Figure 20 .
Figure 20.ETE delay for the MU-IED and CB-IED at the collector bus (Link capacity 100 Mbps, 1 Gbps).

Table 2 .
Measuring requirement for sensor nodes.

Table 3 .
Traffic configuration for a wind turbine.
3.2.Traffic Model for an Offshore Substation Based on IEC 61850

Table 4 .
IEDs configuration in the wind farm substation.IED: intelligent electronic device.

Table 5 .
Traffic configuration for IEDs.

Table 4 .
IEDs configuration in the wind farm substation.IED: intelligent electronic device.

Table 5 .
Traffic configuration for IEDs.

Table 6 .
Locations, types of sensors, and measurement devices installed at the met-mast.

Table 6 .
Locations, types of sensors, and measurement devices installed at the met-mast.

Table 7 .
Wind farm communication requirements based on the IEEE 1646 standard.

Table 8 .
Summary of simulation cases.

Table 9 .
Classification of data types based on different operation modes.

Table 9 .
Classification of data types based on different operation modes.

Table 10 .
ETE delay inside WTs with different numbers of sensor nodes.

Table 11 .
Simulation results of the ETE delay from the wind turbine zone to the control center.

Table 12 .
End-to-end delay of the IEDs at different zones.