Developing Self-Similar Hybrid Control Architecture Based on SGAM-Based Methodology for Distributed Microgrids

Abstract

Cyber-Physical Systems (CPS) are the complex systems that control and coordinate physical infrastructures, which may be geographically apart, via the use of Information and Communication Technology (ICT). One such application of CPS is smart microgrids. Microgrids comprise both power consuming and power producing infrastructure and are capable of operating in grid connected and disconnected modes. Due to the presence of heterogeneous smart devices communicating over multiple communication protocols in a distributed environment, a system architecture is required. The objective of this paper is to approach the microgrid architecture from the software and systems’ design perspective. The architecture should be flexible to support various multiple communication protocols and is able to integrate various hardware technologies. It should also be modular and scalable to support various functionalities such as island mode operations, energy efficient operations, energy trading, predictive maintenance, etc. These requirements are the basis for designing the software architecture for the smart microgrids that should be able to manage not only electrical but all energy related systems. In this work, we propose a distributed, hybrid control architecture suited for microgrid environments, where entities are geographically distant and need to operate in a cohesive manner. The proposed system architecture supports various design philosophies such as component-based design, hierarchical composition of components, peer-to-peer design, distributed decision-making and controlling as well as plug-and-play during runtime. A unique capability of the proposed system architecture is the self-similarity of the components for the distributed microgrids. The benefit of the approach is that it supports these design philosophies at all the levels in the hierarchy in contrast to a typical centralized architectures where decisions are taken only at the global level. The proposed architecture is applied to a real system of 13 residential buildings in a low-voltage distribution network. The required implementation and deployment details for monitoring and controlling 13 residential buildings are also discussed in this work.

1. Introduction

Smart energy systems have been viewed as a typical example of Cyber-Physical Systems (CPS) [1], where the physical and organization processes are carried out, both at local and global scales. Across the world, the electrical grid is going through numerous changes in the transmission and distribution systems. Towards the realization of smart grids, the changes in the transmission grid (high-voltage energy network) are less drastic in comparison to the enhancements in the distribution networks below 10 kV. Some major breakthroughs in the distribution network are the decentralized power generation, increased use of renewable energy generation and integration of storage units [2,3]. These changes mostly affect network districts and ring and mesh networks. Despite the challenges involved in using renewable energy sources in the distribution system, the motivation to use clean energy sources to reduce greenhouse gases is pushing the technological development and regulatory (legal) boundaries. As a result, this has led to the development of advanced technologies [4,5] and novel business techniques. Improvements in the hardware infrastructure include better energy storage systems, smart sensors and advanced materials that can withstand the operational wear and tear. These smart devices are capable of monitoring the real-time information of the environment (e.g., weather data) and can transfer such information over the communication network. The advancements in the communication networks include the advanced communication protocols, the communication channels (Power Line Communication, Ethernet, mobile networks, etc.) and the communication quality features (such as latencies, signal strength, etc.). These advancements in the communication technology facilitate a reliable and robust interaction between the distributed entities.

According to the IEEE Standard for the Specification of Microgrid Controllers (MC) [6], a microgrid is a group of interconnected loads and distributed energy resources with clearly defined electrical boundaries that acts as a single controllable entity with respect to the grid and can connect and disconnect from the grid to enable it to operate in both grid-connected or island modes. A typical microgrid has several consumption devices (also known as loads), as well as the Distributed Energy Generation (DER) units such as solar panels, wind turbines, biogas plants, and combined heat and power plants. DERs are often connected with battery backup systems to make up for the intermittent behaviour of renewable energy generation. During the grid-disconnected mode, also known as island mode, the microgrid uses locally available energy sources [7]. Since the mode of operation depends on several factors such as the energy demand, supply from the renewables, operational constraints of the hardware, etc., a robust MC is required to monitor and control the microgrid.

For harnessing the full potential of renewable energy resources and for supporting stable power in a microgrid, the use of the latest hardware and advanced communication technologies is not sufficient. A Distribution Management System (DMS) [6], which is a collection of applications designed to monitor and control a distribution network efficiently and reliably, is required to the handle advanced hardware and the communication technologies. The design of the DMS involves several innovation areas, such as data processing and data analysis techniques, decision-making using mathematical or statistical models, real-time execution of the control signals, etc. Moreover, several microgrids operating in the distribution network may have smart and non-smart infrastructure and may also vary in size and scale. Therefore, the design of the DMS must be based on flexible, scalable, modular and secure software architecture to support and meet the objectives of the microgrid.

Several approaches for designing the DMS architecture are present in the literature [8,9,10]. They mostly focus on interoperability, flexibility, distributed controlling and on the scalability aspects in microgrids. Chandy et al. [11] proposed a System-of-Systems (SoS) architecture for smart grids. Their architecture focus on the modularity, flexibility and scalability aspects. Most of the literature in the area of the design of DMS architectures either include all the aforementioned aspects but restrict themselves only to the simulation environments or are developed in the real environment while focusing only on a few of the aspects of the microgrid. Practical problems like dealing with weak communication infrastructure and ability to integrate heterogeneous and inexpensive hardware become critical in a rural setting.

Florea et al. [12] proposed a fractal architecture for power grids, using the concept of holons. Holons are the autonomous entities with fixed functionalities and have the ability to interact with other holons. The functionality of each layer in the fractal architecture is the primary, intermediary and tertiary controlling. Holons at the lowest layer interacts with hardware. Holons at the intermediary level act as supplementary generator controllers and holons at the highest level perform smart grid core optimization controls. They focused on two characteristics of fractal systems, namely, self-similarity and self-organization. A centralized control hub holon is simulated and tested to emulate the dialogue with other control holons.

In the direction of fractal design research, there are also ongoing research projects such as Retière et.al. [13], who presented the methodology and initial results of the ongoing project Fractal Grid in the urban areas of Jura and Haute-Saone in France. Their work focused only on the criteria and constraints to be considered for creating the fractals for high and low voltage electrical networks. Initial survey results on the grid management and planning approaches have been discussed. The spatial organization of the electrical grid was considered so the needs of the users located in the urban areas were better addressed and voltage drop was taken as a measure of power quality.

In this work, we address the design challenges for the DMS or Microgrid Controller (MC) [6] for distributed microgrids in rural and remote locations. Based on our prior experience of designing the architecture of an MC for single nodes or smart homes [14], this paper proposes a hierarchical architecture for a distributed microgrid. The proposed architecture is flexible, scalable, modular, capable of handling heterogeneous hardware infrastructure and supports numerous microgrid objectives such as grid balancing, fault management, etc. Although the security and data privacy features are essential for microgrids, these aspects are not directly focused in this work. Architecture is capable of supporting advanced features such as security and data privacy, data analytics, energy efficiency, etc. A basic level of security (role-based authorization and password-based authentication) is supported by the architecture. In addition, data encryption techniques are also used to allow secure data transfer over the communication channel between distributed entities. While focusing on decision-making and intelligent controlling, we propose a self-similar and hybrid control architecture that is embedded throughout the layers of the hierarchy. As proposed by Schaetz [1], the architecture supports the three different dimensions of CPS i.e., self, cross and live dimensions. Self-dimension relates to the local operations required for autonomous operations of a microgrid, such as data handling at different layers and island mode operation in a smart home. Cross-dimension refers to the coordination required among microgrids when connected with the main grid, such as the area level islands, which have multiple smart homes and other loads along with the low-medium generation units. Live-dimension deals with the changes during runtime e.g., switching from grid connected to island mode operations. The proposed architecture is implemented and deployed on a 13-node low-voltage radial distribution system at Amrita University in Kerala, India. The cost of setting up microgrids is usually high and therefore in this work a major consideration was setting up low-cost microgrid in rural areas of India to digitize the electrical infrastructure. This work is carried out as part of the Indo-European research initiative.

This paper is organized in five major sections: Section 2 discusses the existing approaches for MC architecture in both centralized and distributed microgrids. In Section 3, detailed requirements and main design challenges for distributed microgrids are enumerated. Requirements are linked to standards and existing design principles to highlight the applicability and shortcomings of any one single design philosophy. This results in the need for combining different design styles leading to a hybrid control architecture. The proposed self-similar, hybrid control architecture is described in detail in Section 4 where every constituent and fulfillment of requirements are explained. Section 5 provides the implementation details of the proposed architecture and deployment details in real environment. The paper is concluded in Section 7 highlighting contributions and advantages of the proposed architecture. Finally, in Section 6, the lessons learnt, further challenges and future work of the proposed architecture are described.

2. Review on the Architectural Concepts for Microgrid Environments

A massive transformation in the electrical grid is rapidly increasing the possibility of new technologies and concepts. The larger vision and the purpose of the smart grids are relevant to understand the business objectives of the smart grids. In this section, we highlight the need of the architecture concepts that fulfill the business objectives of the microgrid, without depending heavily on the standards and methodology. Farhangi [2] highlights that the system architecture is a key ingredient for smart grids. He emphasizes that the fundamental attribute of future grids is the highly distributed Software Controller (SC), which is capable of monitoring and controlling distributed entities. The SC that is responsible for distributed control should be scalable to handle large amounts of data and offer a wide variety of functionalities. It should be flexible enough to accommodate the latest and future technologies. Amin [15] asserts that overall smartness in the distribution grid can be introduced by adding smartness in each independent entity in the grid. For example, intelligent algorithms implemented independently at device level, energy efficiency included in smart homes and solutions for demand-response at grid operator level, etc. improves the smartness of the overall power grid. He further suggests that this approach not only ensures application of wide variety of solutions but also makes it possible to apply solutions tailored for specific energy needs. Erlinghagen et al. [16] observe that Information and Communication Technology are transforming the electricity sector in Europe. As a result, there is a rise in ICT firms that are coming up with innovative solutions for smart grids. This study highlights that the MC must also support business requirements suitable for microgrids.

In order to fulfill the objectives, the design requirements are essential. Towards creating architecture of MCs, the survey by Martin et al. [17] is helpful, as they provide a clear understanding of microgrids, their requirements, compositions and attributes. These requirements are considered in the proposed design approach as well. Rohjans et al. [18] have systematically worked on the requirements for the SC architectures. They have listed out the essential non-functional requirements (NFR) for SC. The selection of NFRs is according to Sommerville [19].

Chandy et al. [11] highlight the System of System (SOS) architecture approach, which includes architectural styles such as silo, enterprise service bus, adapter principle and Open standard service mechanism, also known as Service-Oriented Architecture (SOA). Although SOA is helpful for distributed applications, the orchestration of services for distributed microgrids is crucial and a real challenge for architecture of MC, which is considered in the proposed architecture. Various software architecture design principles [19] such as Service-Oriented, Client-Server, peer-to-peer, Model-View-Controller and pipes-filters are frequently used in the existing design approaches of MCs, but the usage of one or more design principles depends on the specific microgrid setting. Zhabelova et al. [20] propose multi-agent smart grid architecture based on the communication protocol for intelligent electronic devices (IEC 61850/61499), with a special objective of Fault Location Isolation and Supply Restoration (FLISR), supported by design. The lack of support to other business requirements is missing in their work, such as data visualization, data analytics, predictive maintenance, etc. Towards the system architecture, the most known architecture style for smart grids is Supervisory Control And Data Acquisition (SCADA) [21]. Since SCADA supports only centralized decision-making and controlling, recent efforts have been made to include the distributed decision-making. These efforts include retrieve information from several SCADA systems as servers by a single OPC-UA client using web services. Open Platform Communication-Unified Architecture (OPC-UA) is widely known protocol in the industrial automation community, it is gaining relevance in smart grid research as well. There are also approaches that use the SCADA system with additional control systems. As mentioned before, another system architecture relevant for the SC is holonic architecture style [22,23], originally used for the manufacturing domain. Holons are the autonomous, cooperative and recursive entities that interact with other holons in a holarchic manner. Fractal approach is also proposed for designing smart grid system architecture. Towards the fractal design approaches, emphasis is given on both self-similarity and self-orgnization as described by [12,13]. Taking the fractal concept further, Bytschkow et al. [24] focused on the compositionality of self-similar components. Goal-oriented designing is also another popular approach used by many researchers. Marzband et al. [25] propose Transactive Energy (TE) framework, which comprises many home microgrids. These home microgrids are the individual smart homes or home energy management systems that interact with each other to create coalitions. The basis of creating coalition is the optimal use of installed resources by considering demand fluctuations, energy production, etc. From the design perspective, their work focuses more on the self-organizational aspect.

Not all architectural approaches lend themselves as suitable when applied to microgrids. Various European research projects focusing on the design and architecture of microgrid show the relevance and the insufficiency of the existing solutions. Girbau-Llistuella et al. [26] present the experience and approach used in setting up smart grid pilot network in the rural distribution network in Spain. Towards the distributed controlling, they have created two new agents, communicating with the SCADA system at the Distribution System Operator (DSO) side. In addition, there are many other research projects in Europe, especially focusing on the design of architecture for smart grids [27,28,29,30].

3. General Requirements on the Software Architecture for Distributed Microgrids

In this section, we describe the requirements taken into consideration for the architecture of MCs for the distributed microgrid. As per the functions described in the Standard for the Specification of Microgrid Controllers [6], a microgrid controller must fulfill the following:

• Operation in grid-connected and islanded modes.
• Automatic transition from grid-connected to islanded mode by providing a managed transition to islanded mode for microgrid loads during abnormal bulk power system conditions and planned interruptions of the system.
• Resynchronization and reconnection from islanded mode to grid-connected mode.
• Energy management to optimize, both real and reactive power generation and consumption.
• Ancillary services provision, support of the grid, and participation in the energy market and/or utility system operation, as applicable.

Based on the above requirements, it can be inferred that, in a distributed microgrid, an MC should be capable of monitoring and controlling the distributed entities, which could be real or virtual. Real entities correspond to the physical devices, whereas virtual entities mimic the behavior of physical entities.

From these functional features and, as per the survey conducted by Fang et al. [31] on the future of smart grids, we extract the functional and non-functional requirements for a Microgrid Control System (MCS) [6]. These are listed in

Table 1. Functional and non-functional requirements for microgrids according to the surveys and standards for microgrid controllers. [6,31]

Functional Requirements	Non-Functional Requirements
Distributed control for island mode operations	Modularity
Centralized control for grid-connected operations	Flexibility
Hierarchical structure for coordination and synchronization	Scalability
Plug-n-play to support both features and devices

and described below.

• Distributed control for island mode operations: Several microgrids should be able to operate in an independent manner, when operating in an island mode. These microgrids could communicate with each other without creating any adverse impact on the infrastructure. During the island mode operation, any individual microgrid should be able to balance itself to meet the required power demand at all times. Therefore, microgrids must be monitored and be allowed to run independently.
• Centralized control for grid-connected operations: In case several microgrids are connected with each other in a grid connected mode, these individual microgrids should be able to communicate and share their resources in a collaborative way. Any microgrid can offer to publish certain services, while others requiring these services can subscribe to them in a collaborative manner. In order to access the services, it is essential to know whether the services are internally available (within a microgrid) or external to the microgrid. The internal services within a microgrid perform local actions. For example, intelligent decisions within a smart home. In contrast, several smart homes within a microgrid may offer external services of energy sharing, which can be executed by the node of common coupling.
• Hierarchical structure for coordination and synchronization: When several microgrids combine to form a larger microgrid, synchronization and coordination is required. They can again disjoin to make several distributed microgrids operating in islanded mode. To make this possible, functionalities of microgrids should be organized into a structure. Control functionalities and global controlling should be organized as a hierarchy allowing physical devices to be controlled for both local and global objectives. This can enable a command to be propagated through the hierarchy to all the connected microgrids, e.g., to change to island mode. Additionally, information from individual microgrids can be aggregated and analyzed on higher levels to make a larger grid, supporting a more global decision.
• Plug-n-play: Also known as hot-plugging, this is another crucial requirement for a microgrid. It provides the flexibility to add, remove and modify microgrids or even hardware infrastructure at any time. The microgrid should be able to alter functionalities during island and grid-connected modes. Such plug-n-play features make the microgrid more intelligent and real-time responsive.

In order to highlight the need for the architecture of MC that can incorporate all the above explained requirements of microgrids, we present an association of each microgrid requirement with existing software architecture styles. This association helps in understanding the compatible and incompatible requirements between software design and the microgrid. Thus, the resulting enumeration, shown in

Table 2. Compatibility of functional requirement with the architecture style.

Requirements	Architectural Style
Distributed control for island mode operations	Peer-to-peer
Centralized control for grid-connected operations	Client-server
Hierarchical structure for coordination and synchronization	Layered
Plug-n-play to support both features and devices	Component-based or Service Oriented

, forms the foundation to identify appropriate architecture design patterns suited for the application in distributed microgrids.

The detailed description of the suitable architectural style corresponding to the functional requirement is as follows:

• For the distributed operation of various entities, a peer-to-peer architecture style [19] is better suited, where all peers are able to communicate directly without interfering in the operation of other peers. All the individual microgrids have equal privileges and have direct communication with each other. MCs of individual microgrids are capable of sharing information and are able to switch modes form grid-connected to islands and vice versa.
• A client–server architecture style [19] is suited where a server hosts certain services and multiple clients are able to use these services. The server publishes a service and waits for clients to make requests. When several microgrids switch back to the grid connected mode, the MC for the newly formed larger grid must be able to control the imbalances. Therefore, MC acts as master and the MCs of all the individual microgrids act as slaves.
• The requirement for hierarchical structure of the functionalities is supported by the Layered architecture style [19], where multi-layered design provides abstraction to both data and services. Analysis and planning are done on the higher abstraction layers, as it requires larger response time. Data requirements also depend on the functionalities. Higher layers often have more planning and analytical functionalities resulting in large response times. Lower layers closer to the physical devices are often responsible for the local actions and have shorter reaction times. Most common multi-layer designs have a presentation layer for visualization, a business layer for post processing of data and a device layer for interacting with the hardware in the field.
• To support the plug-and-play requirement, the component-based architecture design suits well, as the functionalities are developed as loosely coupled components. Complex functionalities can be created as composite components. For instance, the smart switching between grid-connected and island mode operations can be applied only when there is a backup energy generation or storage resource available.

There are more requirements which could be addressed in the design of microgrids, but they are either too specific or only slightly influence the architecture design. Component-based architecture and layered design are taken into consideration to fulfill the modularity, flexibility and scalability requirements of the target microgrid. We propose a self-similar, layered and hybrid-control architecture for MC by combining different design styles to support distributed microgrids. This is described in Section 4.

4. Design of a Hybrid, Self-Similar but Distributed Architecture

MCs controlling microgrids both in island and grid-connected mode are required to perform certain functions, as described in the previous section. Consider a scenario, where multiple grids connect with each other to create a larger microgrid. In such a situation, all the MCs of individual microgrids must perform some common core functions such as avoidance of harmonic and voltage fluctuation, etc. Along with these core functionalities, there are local functionalities only relevant within the microgrid. The MC for the resulting larger microgrid now has a new function. Towards such scenarios, we propose a self-similar-, distributed-, hybrid control- software architecture for MCs.

Self-similarity coupled with the hierarchical design helps better cooperation and coordination of microgrids. All three perspectives of self-similarity are embedded in the design of the building blocks namely, structural, functional and behavioral. Structural similarity refers to the replication of the static structure or core functionalities for each individual MC. This also leads to a more robust design consisting of many low cost devices which can replace/take over each other’s functionality on demand. The reuse of core functions in each MC enforces structural self-similarity. Functional similarity refers to the similarity in the dynamic processes. Due to the replication of core functions in each microgrid, certain dynamic processes also remain the same. For example, health check mechanisms, notifications, access control and data storage are the dynamic processes that can be reused. Behavioral similarity refers to the commonalities in the system dynamics. Behavioral similarity is introduced by the hierarchical structure, where each building block has either at least a parent or a child. For instance, behavior of operating in island mode as a single child in hierarchy with a single parent or having several children microgrids, in a grid connected mode.

4.1. Proposed Architecture

For each MC, the proposed architecture can be extended with components to support the objectives of the zone dimension as given by the Smart Grid Architecture Model (SGAM) reference architecture [32]. The interoperability dimension is reflected in the layered design of our architecture, which will be detailed in Section 5.2. In addition, the functionality can be adapted to different SGAM domains like Prosumer, DER, Distribution and Transmission System. In this work, the focus is mainly on the distribution system. This concept has been applied on the different layers in the distributed microgrid.

Considering the non-functional and architectural requirements of the MC, a schematic of the proposed distributed MCs is shown in Figure 1. The distributed microgrid is organized in a hierarchical structure. The structure is composed of layers where each layer denotes the extent of the control i.e., local, regional, global, etc. The global MC provides overall management of the microgrid. It is responsible for the overall network management and imposes global functionalities on the complete microgrid system. A global MC is composed of several intermediary MCs. The intermediary MC is composed of several smaller MCs. MCs operate independently, have their own decision-making capabilities and can communicate with other MCs in the higher or other layers.

The local layer is shown in the bottom of Figure 1. At the local layer are MCs, which are directly communicating with independent DERs and consumption units of a microgrid. Apart from these, sensors and actuators are also connected to the MCs at the local layer. The local MC controls all the devices connected locally within the microgrid. The physical devices provide data to the MC. The data from the physical devices is of two types: either measure of power quality (voltage, current, active-reactive power, etc.) or measure of the environmental factors (such as temperature, brightness, occupancy, etc.). Based on the data, the MC can make decisions to control the actuators. Apart from the capability to control physical devices, the MCs can also perform data analytics based optimization of the connected resources. For example, the decision to operate in an island mode with the help of Photo Voltaic (PV) coupled with battery backup system or to operate in a grid connected mode is a decision that can be taken by the local MC. The local MC operates in a distributed manner and satisfies the idea of edge computing [33] where data processing is closer to the data source, thereby resulting in faster decision-making of resource utilization.

Several local microgrids combine to form intermediary microgrids. These are shown in the middle layers of Figure 1. For example, at the intermediate layer could be streets or districts, which comprise individual buildings. Management of intermediary microgrids is done by the intermediary MCs. The intermediary MC is a collection of the local MCs. It provides a communication channel between individual MCs. For example, the intermediate street MC could balance the power supply between smart buildings and normal buildings. The intermediary MC can also take decisions for all the connected local MCs. Therefore, intermediary MCs play an important role in organizing and clustering towards grid balancing.

The functionalities of an intermediary MCs could include device and data aggregation, which is similar to the aggregation functionalities of a Head-End System (HES) [34]. Device aggregation is the mechanism of combining multiple devices to create a single virtual device. Data aggregation implies combining multiple independent devices to create a single virtual entity, which provides an aggregated single value computed on the data from multiple independent devices. Data aggregation helps in the extraction of knowledge from the raw data provided by devices. In addition to data processing, intermediary MCs provide the capability to generate control actions from automated decisions for predefined objectives. For instance, power theft can be detected by the intermediary MC either by comparing the metered value at the supply point with the aggregated data from the consumers or by aggregating the consumption data coming from the meters installed outside the building. Another kind of information that can be generated by intermediary MC are the Key Performance Indicators (KPI), regarding the performance of a cluster. The intermediary MCs can also be formed by aggregating several other intermediary MCs. For instance, a city is made of blocks, which are made of streets and finally individual buildings. Both blocks and streets could form intermediary MCs.

A single global MC provides overall management of the grid. The is shown in the top layer of Figure 1. The global MC is the final aggregation of the entire available intermediary MC. It is responsible for the overall microgrid management. It monitors and controls all the nodes in the lower layers. For example, at the global level, data processing techniques can be used to determine the performance and health status of the complete microgrid. Statistical data from the intermediary MC such as KPIs can be used to make decisions that affect the entire microgrid. The global MC generates control signals for the microgrid and sends them to intermediary MCs. Intermediary MCs receive control signals and forward these signals to the local MCs. Local MCs convert control signals to appropriate device specific format and send them to the physical devices for final execution of the control command.

In the proposed architecture, self-similarity of the MC enables it to be used for various objectives. For example, any MC at the lower layer can take the role of an intermediary MC (at the higher layer) in case of MC failure at the higher layer. The flexibility of the design allows addition or removal of functionalities. Diverse hardware can be integrated and the architecture is not bound to any particular hardware. The support to scalability ensures the inclusion of both layers and MCs. In other words, the same architecture can be applied to smart buildings, districts, regions or state levels. Modularity is incorporated in the MC by design. Each MC is an independent entity and can be added or removed from the microgrid without affecting the operation of the microgrid. In the case of an island mode, the MC is capable of managing its energy generation and consumption devices.

5. Implementation of Architecture in the Real Environment

In this section, we present the implementation details of the proposed architecture. Following the concepts described so far, a distributed management system was developed and deployed in a real world environment, which consists of a 13 node distribution grid. Each node is connected to a residential district, representing its own microgrid environment. Some districts where equipped with their own energy storage and generators. The architecture is designed for this distribution system and is done in an Indo-European research effort [35] towards integrating solutions from individual technologies starting with an electrical network over a robust communication network and finally a distributed software controller enabling management and control capabilities. This section is divided into three parts: first, the physical and the environmental conditions of the field demonstrator are described. Second, the implementation details of a single MC for a microgrid in the demonstrator are described. In the third part, we describe a hierarchical architecture of the complete demonstration site consisting of self-replicating MCs. Additionally, we explain in detail the similarities and differences of each MC, when placed at various hierarchical levels of the proposed architecture.

5.1. Field Demonstrator

The real world demonstrator site is located amidst the residential buildings in the campus of Amrita University, Kerala, India. The demonstrator on which the ICT architecture is applied is grid-connected with 3-phase power supply of 240 V at 50 Hz.

Figure 2 illustrates the systematic view of the distribution system, with the 13-nodes labeled as buildings. Each floor of the building is considered as a single load point or a power consumption entity and therefore represented as independent box in Figure 2. These buildings are connected with over-head and underground electrical lines. The over-head electrical lines supply power to the buildings, whereas underground lines (depicted as dashed lines) are tie-lines, which are activated only when one or more over-head line(s) are broken resulting in a change of the network topology to restore power supply. Overhead and tie-lines are connected to the electrical poles. At each pole, a smart measuring and a control box is mounted to monitor and control the power consumption of each building. Node 1 is connected to the general power grid (S) on one end and on the other end with the substations in the field demonstrator.

Out of thirteen nodes, eleven consumption entities are residential buildings. A water pump represented as node 12 is connected with node 7. These consumption entities are called as consumer nodes. Building-5 is equipped with rooftop solar panels, labeled as DER in Figure 2. This node is powered by both solar power and power from the grid. It is called a prosumer node as it can both produce and consume power. Other nodes do not have alternative power generation units and rely only on the conventional power supply from the grid.

5.2. Architecture of the Microgrid Controller (MC)

The design of the MC is based on the requirements and architecture principles discussed in Section 3 and Section 4.1. Figure 3 shows the architecture of a single MC. The architecture of MC is based on our previous work [14] for setting up an intelligent node in the office environment.

Components are organized as low-level, core, high-level and external components. Hierarchical structure is established by implementing device interfaces in the low-level layer also known as a device layer, in which physical devices are integrated. The low-level functional components (or device interfaces) interact with hardware including virtual peripherals. For instance, in the 13-node distribution system, smart meters are installed at each building. Voltage, current, active power, reactive power, etc. are measured by smart meters, which are integrated as actuatorclients software components in the MC. The smart meters are also connected with the switches, which can be controlled by the actuator client component. The feedback of the change in the switch status is read using I2C serial protocol to ensure that the control command generated translates into a real hardware control signal.

The core-component layer has components that carry out data handling, data organizing and data persistency functionalities. These core-components must be implemented for each MC to carry out specific core objectives. By the use of core-components, replication is introduced towards self-similarity. Specific objectives of each individual MC are implemented in the high-level components. Examples of each individual specific objective could be forecasting consumption or generation using data analytics, optimization of energy, etc. These high-level components are optional and do not affect the operation of core components and hence can be added, removed or modified anytime, providing plug-n-play capability to the MC. For coordination among MCs, each MC can communicate with other using external components. The external interfaces provide communication to the entities which use the data generated by the individual MC. Few examples are interaction with other MCs, offering information conforming to standards like IEC 61850 or Device Language Message Specification (DLMS), information sharing with energy markets or enabling services for network operators like demand side management.

We provide a quick overview of each core component shown in Figure 3. Core components are mandatory for the operation of single instance of the basic unit:

• InformationBroker provides the data logging/persistency functionality. This component provides several interfaces to common backend storage technologies such as MySQL, PostgreSQL, MongoDB, etc.
• ContainerManager models the real world entities/physical infrastructure by defining the unit’s relationship in the existing physical setup. In the previous work [14], we described the smart office building structure (floor, rooms, devices), where a building is the root container, which has several floor child containers. Each floor has several rooms and rooms contain several devices as children.
• UserManager provides the access control functionality for the operation of the unit by defining roles and authorization mechanisms for the authorized/restricted access.
• Ambulance acts like a watchdog for monitoring health of each component and making the user aware by sending notifications using the postman component. This component initiates the fault detection and handling techniques internal to the software application.
• Postman provides interface to various notification technologies such as alarms, emails, tweets and SMS, etc.
• Remoteframework Library connects with the message bus, acting as a communication channel between the components to interact with each other.
• SMGConf acts as a configuration library containing information of the addresses and ports for databases and device specifications.

The proposed architecture enables the replication of MC for a single microgrid and provides the ability of distributed decision-making among the participating microgrids. Therefore, self-similarity is embedded in the architecture of each MC.

5.3. Implementation of a Microgrid Controller (MC) Following the Proposed Hybrid Architecture

The proposed hybrid architecture (Figure 1) adopted to the 13-node system (in Section 5.1) is shown in Figure 4. The 13-nodes are organized as a set of so called Intelligent nodes at the local control layer, three aggregators at the area control layer and a single control station at the global control layer. All the intelligent nodes have physical devices (smart meters) connected with them, which are installed, inside the smart measuring box. Each area is monitored and controlled by a unique aggregator. Note that the number of independent areas is proportional to the number of aggregators. This information is processed and forwarded to the control station. As highlighted before, the aggregators are also capable of executing commands or accessing information depending on their configuration.

Aggregator-2 receives consumption (Voltage, Current) from the Intelligent nodes 3, 4, 5, 6 and 7 to generate KPIs. These KPIs are the indicators of the power quality supplied at the intelligent nodes summarized as average, maximum or minimum daily consumption of each intelligent node.

Figure 5 shows the 13-node distribution system from the ICT perspective. In the cyber infrastructure, the 13 nodes are shown, using the Raspberry Pi deployment platform. A Java-based application based on the MC design as shown in Figure 3 is deployed on each Raspberry Pi. There is a wired connection between every Raspberry Pi and smart meter. The communication between nodes and aggregators (represented as AGG-1, AGG-2, AGG-3) is currently using REpresentational State Transfer (REST)-based web interfaces [36]. These interfaces are flexible and, in the future, these can be replaced by appropriate Standards like DLMS, IEC 61850, etc. depending on the site requirements. The three aggregators communicate with a single control station.

Figure 6 provides a combined view of the internal architecture of a distributed microgrid, MCs placed in different layers and the communication channel across layers. Tracing the communication across the three layers between intelligent node-6, aggregator-2 and the control station, Figure 6 emphasizes the self-similarity of the proposed hybrid architecture. For the sake of simplicity, the detailed view of core components have not been shown as they are the same as in Figure 3.

At the bottom of Figure 6, local control layer shows components of node-6. The physical devices connected with Intelligent node-6 are: photovoltaic, battery and smart meter(s). Devices are connected as actuatorclients, which implement appropriate communication protocols and logic to convert the device-specific information into a homogeneous format. The high level functional components for node-6 are a rulesystem and an island mode operation component. As the name indicates, the rulesystem provides the capability to make smart decisions based on the available information. Constraints are developed as set of rules in the rule system. Island mode operation means the actions carried out to use either solar power or the stored energy from the battery. For communication with aggregator-2, REST calls are made to send the device data and to receive the control signals either from aggregator-2 or from the control station. A REST interface at each layer provides the flexibility of external communication such as visualization of raw data at node level and KPI data at the aggregator level.

The raw data from node-6 is sent to the aggregator. The high level components of the aggregator are a rulesystem, a power theft detector, area mode operation and a KPI generator. The functionality of the rule system is the same as at node-6, i.e., smart decision-making. Power theft detection is based on the consumption values from all the connected Intelligent nodes (3, 4, 5, 6 and 7) and then checking the power consumption values with pre-defined threshold values. KPI shows the quality of power supplied provided to Intelligent nodes. These quality indicators are Voltage fluctuations, harmonic distortions, daily peak consumption, etc. The Area node operation enables power sharing from photovoltaic at node-6 with the adjacent nodes. For communicating with the control station, REST calls are made to send the generated KPIs and to receive control signals from the control station.

The global control layer is shown as the top layer in Figure 6. The KPI data and the regular backup data from the aggregator are received by the control station and saved in the local database. The high level functional components include global decisions concerning the complete network such as network reconfiguration, KPI storage, visualization and rule system. Similarly, post processing is done on the stored KPI.

Finally, Figure 7 shows the complete implementation of MCs in the 13-node system. This 13-node system is in operation since March 2018 and data is being generated and stored at regular intervals on the control station server, where post processing can be done for statistical analysis or to support energy related services.

6. Discussion

As mentioned above, in this work, the architecture concept to support low cost equipment to fulfill the complex tasks in a distributed microgrid is established. A robust underlining architecture enables the infrastructure which supports relevant services of microgrid. The architecture of intelligent nodes, aggregators and control station is related to the general trend of edge computing. Shifting more processing and decision processes closer to the embedded devices results in increased scalability. Another benefit of the decision-making at the intelligent nodes level results in lower upstream bandwidth requirement for the communication network. In situations where the communication networks are unstable, like at the implemented site, decision-making at the intelligent node is clearly advantageous. Unlike a classical hierarchical network, where the control signals are generated only at the highest layer, the proposed architecture offers decision-making and controlling at all the layers. Self-similarity enables on-the-fly change of roles. In case of a failure at a higher-level component, e.g., failure of an aggregator, one of the intelligent nodes can take over the functionality until the aggregator is restored or replaced. Moreover, the proposed self-similar building blocks support additional energy services for data preprocessing and decision-making. The proposed architecture has been implemented in a rural setting in India, where communication network and weather conditions were major constraints. The stability and signal strength of the communication network was a challenge. Appropriate communication networks were selected based on the detailed investigation of impact of weather conditions and other infrastructure constraints. This is out of the scope of the paper and not described here. Therefore, use of only a single communication protocol was not feasible in our work. A wired connection exists between the smart sensors and the local MCs, while a wireless communication channel is present between intelligent nodes at the local layer and the aggregator MCs.

Further investigation is required for the seamless operation of microgrids. An apparent limitation is the communication network connection issue; if the aggregators are not very well interconnected, then intermittent communication network failures may cause partial or total information loss of dependent building blocks. Therefore, interdependency among the self-similar blocks must be taken into account at the design time to cope up with the communication network glitches. Another consideration is maintenance of such systems. Each residential owner should have the capability to maintain and modify the local MC, but with the increase in the system complexity and higher modularity, maintenance and ownership of application and information becomes an additional challenge. It becomes increasingly difficult for multiple owners of the same application to maintain the complete system. These aspects are worth investigating from the maintainability perspective. In this work, the ownership and maintenance of the application is done by a single entity i.e., Distribution Station Operator. Lastly, the integration of self-similar blocks using heterogeneous services requires highly skilled developers and might require considerable time and effort. These limitations were out of scope of this research work and require experts from different areas of research.

The architecture concept for the demonstrator allows flexibility for future extensions such as the integration of additional buildings or floors. To scale the system, the existing intelligent nodes can be replicated and connected to the existing or new aggregators. The integration of an additional hierarchical layer of sub-control stations managing a subset of the whole network is feasible. Aside from the extension by additional adding nodes, the functionalities of the respective subsystems can be extended too. The fine-grained monitoring and control capabilities of the deployed system allows implementation of additional analytical and visualization services to increase energy awareness, replacement of components and dynamically changing the role of any MC.

As the next step in this work, data analysis and optimization techniques will be carried out to assess the performance of the system. Based on the data availability, various intelligent algorithms like network reconfiguration, frequent switching from island mode to grid-connected mode and vice versa will be tested. Graceful degradation strategies will be implemented in order to maintain a grid-disconnected condition for as long as possible. The system is also planned to equip smaller subnetworks with PV and battery backup systems to operate in an island mode. Here, each island could be responsible for the operation of a subnetwork and for interacting with the neighboring subnetworks.

7. Conclusions

In this paper, one of the application domains of Cyber Physical System is discussed. A distributed, hybrid control architecture is proposed for smart micro grids. In the proposed architecture, various functional and non-functional requirements are satisfied and a direct relationship is established between the distributed microgrids and design concepts of software architecture. The distinct contribution of this work is the use of fundamental software architecture concepts for a complex microgrid application domain. This work demonstrates the need of software engineering concepts towards realization of CPS. Peer-to-peer architecture is observed to be relevant for distributed control for island mode operations. Similarly, client–server architecture for centralized control, layered architecture for hierarchical structure and component or service-oriented architecture for plug and play of features and devices are chosen. The distributed microgrid is organized in a hierarchical structure, composed of layers. The layers could be local, regional, global, etc. Architecture of each individual MC is also presented. MC architecture is replicated for every node, at every layer of the hierarchy. The MCs in local layers communicate and interact directly with the DERs and consumption devices of the microgrid via smart sensors and actuators. Multiple local MCs can be aggregated to form an intermediate MC, which can take decisions for all connected local MCs. The intermediate MC further acts as a communication channel between local MCs and the global MC. The global MC is the final aggregation of all individual MCs and provides overall management to the entire microgrid. It also can take decisions that affect the entire microgrid. The proposed architecture has self-similarity embedded at every layer of the hierarchy. Components in the self-similar building blocks could be classified as low-level, core, high-level and external components. Low-level components interact with the physical hardware. The core components are the same in every MC irrespective of the layer in which it is present. The core components carry out data handling, data organizing, and data persistency functionalities. Layer specific objectives are included in the high-level components. The high-level components are optional and do not affect the working of the core components. Interfacing with other MCs is done using external components. By using a common structure for every MC, replication is introduced towards self-similarity. This enables flexibility, modularity and scalability required for the local and global scale operations. Towards the evaluation of the architecture, it is implemented in an actual 13-node radial distribution system at Amrita University Campus, India. The details of the implementation and the challenges faced are discussed in this work. Data from the implementation site is being collected since March 2018. The implemented architecture satisfies both the functional and non-functional requirements and thereby demonstrates the relevance of the proposed architecture for distributed microgrids.