The Application of Ontologies in Multi-Agent Systems in the Energy Sector: A Scoping Review

Abstract

Multi-agent systems are well-known for their expressiveness to explore interactions and knowledge representation in complex systems. Multi-agent systems have been applied in the energy domain since the 1990s. As more applications of multi-agent systems in the energy domain for advanced functions, the interoperability raises challenge raises to an increasing requirement for data and information exchange between systems. Therefore, the application of ontology in multi-agent systems needs to be emphasized and a systematic approach for the application needs to be developed. This study aims to investigate literature on the application of ontology in multi-agent systems within the energy domain and map the key concepts underpinning these research areas. A scoping review of the existing literature on ontology for multi-agent systems in the energy domain is conducted. This paper presents an overview of the application of multi-agent systems (MAS) and ontologies in the energy domain with five aspects of the definition of agent and MAS; MAS applied in the energy domain, defined ontologies in the energy domain, MAS design methodology, and architectures, and the application of ontology in the MAS development. Furthermore, this paper provides a recommendation list for the ontology-driven multi-agent system development with the aspects of 1) ontology development process in MAS design, 2) detail design process and realization of ontology-driven MAS development, 3) open standard implementation and adoption, 4) inter-domain MAS development, and 5) agent listing approach.

1. Introduction

The energy sector is facing a new paradigm shift following the large-scale integration of renewable energy sources (RES) [1]. The significant use of fossil resources is one of the major concerns of today’s society. Climate changes, environmental impacts, and the scarcity of resources have led to the need for RES. RES reduce greenhouse gas emission while contributing to an increase in life quality and sustainable development [2]. The inclusion of RES is a highly complex task. The demand and supply need to be balanced due to the unpredictable behavior of RES. This influences not only the electricity system but also heating and cooling systems due to the considerable linkage between subdomains.

In order to solve these problems, multiple stakeholders need to work together and provide solutions. Models of such solutions are essential to explore the interactions between consumption, production, and transportation as well as economic, environmental and technical phenomena. Multi-agent systems (MAS) can contribute to explore and develop such solutions since MAS can simulate how multiple stakeholders work, interact, and influence each other. The MAS simulations make it possible to simulate systems which consist of agents with different or conflicting objectives.

Agents often collaborate towards a specific goal and need to communicate and share results. Different languages and vocabularies are domain-specific, and often cause problems for the agents in a system. It requires a common language to ensure that messages are interpreted correctly between agents [3]. Therefore, ontology can be applied to establish effective communication between agents. Ontology can specify terms that are used for communication within a specific context and enable agents to make declarations or ask queries that are understood by all other agents in the system [4]. It is an important tool for the development of an intelligent multi-agent energy system, e.g., for the knowledge sharing and knowledge reuse [5].

As more applications of multi-agent systems in the energy domain for advanced functions, the interoperability challenge raises due to an increasing requirement for data and information exchange between systems. Meanwhile, the energy system is strongly connected with other domains. Therefore, the application of ontology in multi-agent systems needs to be emphasized and a systematic approach for the application needs to be developed.

Although some review papers have investigated agent-based modeling and tools for the electricity domain (e.g., [6]), very few studies have investigated the MAS design and the applications of ontology in MAS for the energy domain. Moreover, many studies focus on specific subdomains and how to solve one specific problem. Hence, investigation and analysis of more complex systems and problems, integration of subdomains, including different agents and ontologies, is needed. Meanwhile, it is important to highlight the relevant literature and map the key concepts underpinning the research area [7]. The scoping review can provide the means that identify, characterize, and summarise existing literature regarding the state of research activities. Moreover, the review result can identify gaps in the literature.

This paper conducts a scoping review to investigate the existing studies on the application of ontologies in the MAS for the energy domain. Based on the results of the literature analysis, this paper proposes a recommendation list for the ontology-driven MAS development for the energy domain. This recommendation aims to address certain aspects that are missing in the literature or need more emphasis in future work.

The paper is organized as follows: Section 2 describes the methodology and the research process. Section 3 presents the literature analysis results, and Section 4 discusses the findings followed by Section 5 that concludes. The conclusion section also states the recommendation for future work and the limitations of this study.

2. Method

The study is designed to compile the relevant contributions from previous publications and to analyze their results in relation to multi-agent modeling design for the energy domain. This study firstly conducts a literature search of ontologies and multi-agent systems for the energy domain. The literature search was performed during the first quarter of 2019. To retrieve the relevant articles for this literature study, four online databases are selected that are relevant in the fields of energy, and MAS and ontologies: ACM digital library, IEEE Xplore, Web of Science, ScienceDirect. The review covers books, conference proceedings, academic journal articles, research articles, and review articles. Other forms of publications, such as newspapers, posters, etc., were not considered since their publication forms are not for scientific research purposes. There was no limitation on the publication years for the literature search.

The data collection was divided into three rounds with relevant keywords. The keyword search was only applied to titles due to a large number of the literature in the fields and the concerns of the relevance in the selected domains. The first round focused on the multi-agent systems in the energy domain. To avoid excluding any relevant study, the search strings were:

(‘multi-agent’ OR ‘multiagent’) AND (‘energy’ OR ‘electricity’ OR ‘heating’ OR ‘grid’ OR ‘electric’ OR ‘power’ OR ‘wind’)

The strings, in the first round, resulted in 1433 publications. The result from each database is shown in

Table 1. Results in the first round search.

Database	Result
Web of Science	355
IEEE	822
ScienceDirect	58
ACM	198
Total	1433

. All these 1433 publications were imported to the reference management software- Endnote (https://endnote.com/).

To dismiss the duplicated publications, i.e., articles which were obtained through multiple databases or strings, 856 articles were removed by this criterion. The remaining 577 articles were selected for further analysis. This study searched the remaining articles with ‘ontology’ OR ‘ontologies’ in titles, abstracts, and keywords, and resulted in 24 articles with full-text.

Based on the text mining in the analysis software NVivo (https://www.qsrinternational.com/nvivo/home) and careful review, the 24 articles were separated into six sub-domains (shown in

Table 2. Six addressed sub-domains by the selected articles.

Grid Control	Power System	Energy Management System	Microgrid	Buildings/Demand Side	Electricity Market
8	3	2	3	5	6

). Majority of the selected articles only address one sub-domain, and one article [8] addresses three sub-domains (energy management, microgrid, and buildings), and another article [9] addresses two sub-domains (power system and microgrid). The publications show that the application of ontologies in the field of MAS for the energy domain was mainly conducted after the year 2004, with focus on the sub-domain of grid control between 2004 to 2014, and expanded into the sub-domain of electricity market since 2014. A list of the 24 articles in the Appendix A shows the focused aspects in the energy domain, ontology, and MAS design.

3. Results

This study reviews and analyses the selected 24 articles to investigate the current research on the application of ontologies in MAS for the energy domain, and the main discussion in the 24 articles can be divided into five categories: 1) definition of agent MAS, 2) MAS applied in energy domains 3) defined ontologies in the energy domain, 4) MAS Design and architectures, and 5) Ontology in the MAS development.

3.1. Definition of Agent and MAS

3.1.1. Agent and Agent-Based Modeling

An agent is defined as an entity that reacts to changes in its environment through a reasoning process [10]. The attributes of an agent are autonomy, sociability, reactivity, pro-activeness, adaptiveness, interactivity, rationality, and interactivity, etc. [11]. Russell [12] defines an intelligent agent as an autonomous entity which has the following properties:

• It has the ability to communicate and interact with its environment;
• It is able to perceive the (local) environment;
• It is guided by basic objectives;
• It has feedback behavior.

An agent structure shows: (1) a set of modules that the agent is decomposed in to, (2) the interaction between these modules and the environment and other agents (shown in Figure 1), and generally, there are three types of agent structures: deliberative architecture, reactive architecture and hybrid architecture [13].

Agent-based modeling is a model of a system with the description of agents and agents’ interactions [14]. Agent-based modeling usually models part of the system rather than a whole system due to the complexity of the system.

3.1.2. Multi-Agent Systems

Multi-agent System (MAS) is a complex system that is composed by more than one distributed agents and these agents communicate to deal with problems which usually can’t be solved by a single agent [14,15]. According to [16], a MAS is characterized by:

• Large numbers of actors are able to interact, in competition or in cooperation;
• Local agents focusing on local interests and negotiating with more global agents;
• Implementation of distributed decision making, through negotiation processes between different local or global agents;
• Communication between actors is minimized to generic information exchange between agents: only the information necessary for their functioning is sent between agents.

MAS is based on the divide-and-conquer mechanism [17]. In a MAS, each agent has limited knowledge about its environment, and work individually towards a certain goal based on their local knowledge and their behavioral algorithms and interact in a cooperative or competitive manner with other agents [18].

The idea of using MAS is to divide a complex system into smaller and more related objectives and construct agents for these sub-objectives [17]. MAS can simulate and control large complex decentralized systems that can cope with the dynamics of the system, reduce the complexity, and increase flexibility [19]. One of the most important benefits of MAS is its fault tolerance, based on multiple agents can provide the same services [17].

3.2. MAS Applied in Energy Domains

The energy sector is becoming more complex and consists of multiple hybrid systems, which includes various interactions and amounts of knowledge. MAS is being studied in many areas of power engineering including diagnostics, condition monitoring, power system restoration, market simulation, network control and automation, and hierarchical decision making, as smart grid (SG) and microgrids (MG) [18,20]. The development of simulation platforms based on MAS is increasing as a good option to simulate real systems in which stakeholders have different and often conflicting objectives [21].

3.2.1. MAS for Grid Control

According to [18], research on using MAS in power engineering mainly focuses on distributed control architectures and simulation. MAS is a decentralized scheme that utilizes distributed controllers for energy management and optimization, and it is an alternative approach for smart system optimizers (SSOs) implementation within a typically integrated energy system (IESs) [22]. MAS is an obvious and promising choice for the smart grid control system because MASs can overcome the threat of SPOFs (single-point-of-failure) due to their distributed characteristic [23]. Meanwhile, Considering the agent properties, the variety of components used in power transformer and the huge amounts of data involved, MAS provides the best possible choice for the purpose of monitoring, automating, controlling and diagnosing the power transformer components [24]. MAS has proven to be suitable for addressing the demands of SGs both theoretically and practically [25].

Most of the research work in this area have focused on hierarchical control, optimization, and power restoration using MAS. For instance, [21] proposes a MAS-based optimal energy management solution for the optimization problem of the interactive operation of generation units and DR [26]. Similarly, introduces a decentralized agent-based approach for optimal residential demand planning [27]. A MAS is used in [28] to restore power in case of failure, and [29] introduces a flexible and versatile MAS for fault isolation and power restoration. Meanwhile, [30] presents a MAS automated management and analysis of SCADA and Digital Fault Recorder Data. Furthermore, a multi-agent system is used to control the voltage of the power system with co-ordination in [31].

Other distributed MAS-based solutions to grid control are also presented microgrids, islanded microgrids, and multiple microgrids [8]. The applications of MAS in a microgrid is similar to the smart grid control, e.g., Microgrid control, optimal energy exchange, and multi-level management, but also link to buildings or demand-side management. For instance, [32] presents a MAS for Microgrid control and a classical distributed algorithm. [33] proposes a MAS microgrid system for optimal energy exchange between the production units of the Microgrid and local loads. based on MAS, [34] proposes an Intelligent Distributed Autonomous Power System (IDAPS) to increase the reliability of the critical loads. [35] proposes a multi-level management and control scheme for microgrid systems taking into account the interaction among agents at different levels. [36] presents a consumption scheduling framework in small residential areas.

3.2.2. MAS for Electricity Markets

MAS of the electricity markets concern market players and markets modeling, strategic bidding and decision support [37]. Multi-agent-based simulation of the electricity markets usually combines with artificial intelligence techniques and game theories and is not only simulation platforms but also provides opportunities for the scenario comparison, future evolution study and sensitive analysis [38].

Several studies have applied MASs to model and simulate electricity markets [14]. For instance, Li et al. [39] discuss the potential for developing Open Source Software (OSS) for power market research. The Agent-based Modelling of Electricity Systems (AMES) is an agent-based OSS laboratory, specifically designed for the experimental study of reconstructed wholesale power markets. The AMES simulation includes an independent system operator, load-serving entities, and generation companies distributed across the transmission grid.

Another electricity market model is the Electricity Market Complex Adaptive System (EMCAS) model [40] utilized by Koritarov [1]. The model is used to capture and investigate the complex interactions between the physical infrastructures (generation, transmission, and distribution) and the economic behavior of market participants [41]. Furthermore, the model applies an agent-based approach where agents’ strategies are based on learning and adaption. This approach enables simulations in different time periods, from real-time to decades including both pools and bilateral contract markets. This approach also makes it possible to see the evolution of an electricity market over time and stakeholders’ reaction towards changes in economy, finance, and regulation. The study describes two methods of how the agents learn: observation-based and exploration-based learning. In observation-based learning, the learning process is based on a structured process of past market performance evaluation, future market status prediction, and investigation of other agents’ actions. Agents decide either to keep or adjust their current market strategy or use a new strategy. Agents based on exploration-based learning explore new market strategies, and these strategies are simulated in a simulation tool. The results are observed, and the strategies are either accepted or rejected based on the results and the agents’ goals.

Praca et al. [42] develop the Multi-Agent Simulation of Competitive Electricity Markets (MASCEM) [43]. The model is developed to study the behavior and evolution of an electricity market. The MASCEM is a modeling and simulation tool aiming to study the operation of complex and competitive electricity markets [44]. The agents in the system represent the market entities, such as generators and customers. The MASCEM allows agents to establish their own decision rules and adapt their strategies as the simulation progresses based on previous events. As a decision-supporting tool, the simulator includes different possibilities regarding electricity market negotiations [45,46]. The MASCEM is a flexible tool which makes it easy for users to define models including strategies, types of agents and market types. For example, this flexibility is utilized by Santos et al. [3,47,48] for modeling and simulating the EPEX (central European electricity market) and Nord Pool spot market (Scandinavian electricity market). The MASCEM can also be used for modeling and simulation of other electricity markets such as MIBEL (the Iberian electricity market), GME (the Italian electricity market), and even markets outside Europe [48].

3.2.3. MAS for Demand-Side and Building Systems

MAS provides a flexible and reliable solution to manage and optimal loads at demand-side with the consideration of energy cost minimization and user’s comfort maximizations [49,50]. MAS has been applied in automated building management systems (BMS) for energy-related building research [16,51,52,53].

The automated BMS research in energy-related building systems mainly focuses on control mechanisms of building loads and investigate possibilities and potentials of energy efficiency and flexibility in buildings [54,55], and especially much equipment in buildings can be controlled and deliver demand flexibility, e.g., lighting and HVAC, and can respond to the grid signals [56]. Although complex control systems are important in building systems, these processes need to be optimal, flexible, and automated.

Multi-agent-based modeling techniques have been used to integrate real-time intelligent decision-making in building control. For instance, an indoor environment that actively supports its inhabitants can be created with these techniques [57]. These modeling techniques also include unpredictable user-behavior, fluctuating weather conditions, and grid imbalances [52,58]. For instance, the study by Anvari-Moghaddam et al. [52] demonstrates how MAS is used to optimize management strategies for a building through computer simulations in combination with third-party software such as MATLAB and GAMS. Hence, studies show that energy consumption can be reduced without compromising the inhabitants’ comfort level in residential buildings.

In the study [52], a smart grid is simulated with several residential buildings, conventional and RES. The residential buildings include underfloor heating, heat pumps, and energy storages. The simulation incorporates meteorological data for the examined location together with technical data, to estimate the power production from RES. The simulation result shows that it is possible to reduce domestic energy consumption and meet the system’s objectives and constraints at the same time. However, the study does not take fault-tolerant and uncertainty handling capabilities into account.

The study by Zeiler and Boxem [16] analyses how smart grid and building optimization can work together and presents an ontology of a software system which acts as a bridge between BMS and a smart grid. Several experiments are conducted in this study to test a HVAC system in a building environment, including the interaction with a smart grid. The study also includes the dynamic behavior of the occupants towards the systems in combination with an overall goal of energy efficiency. The study finds that different elements depend on each other, e.g., changes in required heating affect the available energy. The automated equipment, controlled and managed by the building, responds to demand response requests from the grid to balance the grid condition [59]. The experiment also shows that the comfort level increases while the energy consumption decreases in their MAS modeling.

Meanwhile, the study by Mousavi et al. [53] includes the unpredictable nature of the business process in an office building in a simple model with only a few devices to control. This study does not include a response to the grid conditions. Instead, the study investigates an energy automatic model for office buildings to reduce energy consumption and increase the indoor comfort level. The model is a MAS with the ontology based on the standard IEC 61499 (automation system standard) [60]. The goal of this study is to optimize the energy consumption in an office building where the ontology provides the communication logic and allows agents in the model to share knowledge and data [61]. In the MAS model, agents communicate and collaborate towards a common goal. The method has been applied to an office meeting room, where meeting activities and equipment can be automatically controlled, including measurements of energy consumption. Based on the data gathered as a result of the simulation, the study shows that it is possible to reduce 50 % of the room’s monthly energy consumption by controlling the operation and preparation of the room automatically. The duration of the meeting room simulation is 20 working days (1 working month). The simulated BMS automatically acknowledges the meeting schedules and needs for shading, screen, and blackboard usage, etc. The business process is combined with automated processes to overcome the inefficient use of energy in buildings and lower the number of system failures.

3.2.4. MAS Tools for the Energy Domain

In a MAS of the energy system, agents can represent market players, network components, or part of/a whole system [9]. Therefore, the multi-agent architecture of energy and power systems is designed for dealing with the system complexity [9,23]. Meanwhile, multi-agent simulations allow investigating the statics and changes of the physical systems, electricity market and market players’ behaviors. There are multi-agent simulators in the various domain for different purposes, e.g., CoABS (https://www.cs.cmu.edu/~softagents/project_grants_coabs.html) grid [62]. The selected literature shows that the multi-agent simulators in the energy system can be divided into three main areas:

• Multi-agent simulators for smart grid:
  • Mosaik (https://mosaik.offis.de/): [49,50] is a flexible smart grid co-simulation framework, and allows to reuse and combine existing simulation models and simulators to create large-scale smart grid scenarios [63]
  • MASGriP (Multi-Agents Smart Grid Simulation Platform): models the internal operation of a smart grid with the consideration of all involved players [21].
• Multi-agent simulators for the grid communication, monitoring, and control:
  • Electric Power and Communication Synchronizing Simulator (EPOCHS) (http://www.cs.cornell.edu/hopkik/epochs.htm): aims to solve network communication problems and avoid potential costs and damages by the combination of the results of several simulators [64].
  • Global Event-Driven Co-Simulation framework (GECO): models and simulates the control, monitoring, and protection of the power systems and communication network [65].
• Multi-agent simulators for electricity markets:
  • Multi-Agent Simulator for Electricity Markets (MASCEM) (http://www.mascem.gecad.isep.ipp.pt/overview.php/): can simulate many market models and player types, and enable decision-support [21].
  • Agent-based Modeling of Electricity Systems (AMES) (http://www2.econ.iastate.edu/tesfatsi/AMESMarketHome.htm): simulates wholesale power market operation including load, market participants, grid [66].
  • Power Trading Agent Competition (Power TAC) (https://powertac.org/): is an open-source platform that simulates future electricity market including broker types of energy retailers, commercial or municipal utilities, or cooperatives [67].
  • Electricity Market Complex Adaptive System (EMCAS) (https://ceeesa.es.anl.gov/projects/emcas.html): simulates diverse participants’ strategies and behaviors in the electricity market [68]
  • Multi-Agent Negotiation and Risk Management in Electricity Markets (MAN-REM): simulates electricity markets, and emphases the bilateral contracting and risk management [37].
  • Adaptive Learning strategic Bidding System (ALBidS): aims to integrate market strategies, evaluate performances under different contexts of negotiation, and provides decision support to electricity markets negotiating players [69].

3.3. Ontology and Defined Ontologies in the Energy Domain

3.3.1. Definition of Ontology

The term ‘ontology’ is originally introduced by the Greek philosopher Aristotle [70] as a theory about the nature of existence. Since the beginning of the 1990s, ontology has been adopted by information scientists in the field of artificial intelligence and web and system modeling [71]. In computer science, the ontology is defined as: “a formal, explicit specification of a shared conceptualization.” [72]. This explicit formal specification is domain-specific [73]. Ontology provides a model to support the process in agreement with all parties that all parties commonly agree to refer to the ‘specification’ of a conceptualization [74]. Uschold [75] identified different categories of ontologies:

• Communication between people. Here, an unambiguous but informal ontology may be sufficient.
• Inter-operability among systems achieved by translating between different modeling methods, paradigms, languages and software tools;

In the Artificial Intelligence community, ontologies describe entities and their properties, relationships, constraints and behavior that are not only machine-readable but also machine-understandable [14,24]. According to [13], the functions of ontology are:

• Communication: ontology can provide common glossaries to communication among different individuals.
• Interoperation: ontology can freely interpret and map among various modeling methods, languages and software tools.
• Reuse: the ontology’s analyses clarify the structure of the field’s knowledge in order to lay a good foundation for knowledge representation. Ontology can be reused, so the repetitious knowledge analyses can be avoided.
• Knowledge acquisition and sharing: to construct the system based on knowledge, the available ontology can be used as origination and foundation to supervise the acquisition of knowledge, which can improve its velocity and reliability.

To build an ontology, knowledge engineers need to talk with domain experts to analyze the system and to make everything explicit, e.g., concept description with existing defined concepts and the knowledge rules (i.e., the decision-making rules) in these formalized concepts [76]. There are seven recommended steps to design an appropriate ontology. The developed ontologies provide the means to exchange information that can be interpreted by software agents, knowledge representation and sharing among the software agents [38]. Ontologies are also useful for sharing between modelers, domain, experts, and users [14]. Meanwhile, ontologies also enable to infer knowledge from the gathered information using a reasoner [38].

Many languages have been developed to build an ontology for different purposes. The Ontology Web Language (OWL)(https://www.w3.org/OWL/) by W3C is one of the most popular standard ontology languages. It possible to use OWL in a variety of applications such as knowledge sharing and representation [77], semantic web [78], information system [79], ontology-based reasoning [80], etc. An important requirement for the system interoperability is to reuse existing ontologies. There are some libraries of reusable ontologies available online, such as Ontolingua (http://www.ksl.stanford.edu/software/ontolingua/) and DAML ontology libraries (http://www.daml.org/ontologies/) [48].

3.3.2. Defined Ontologies in the Energy Domain

There are some ontologies already developed for specific energy domains [81,82]. For instance, Kofler et al developed an ontology that focuses on energy consumption and energy provision [83]. [84] proposes 6 basal ontologies for energy management system:

• Cognitive ontology: the activity that agents analyze power systems.
• Physical entity ontology: the equipment that is used for transmitting electric energy and its connecting topology.
• Data ontology: the magnitude that cognitive agent has apperceived to respond to physical entities.
• State ontology: the generalization of the current operation mode in an electric power grid.
• Event ontology: all aspects that create changes of state.
• Operation Ontology: the combination of all actual actions that a cognitive agent does on physical entities.

The well-described ontologies in the energy domain are mainly found in the electricity market domain. For instance, [85] develops an ontology for the electricity market named Electricity Market Ontology (ELMO). It provides a shared, common understanding of concepts and procedures in the electricity market operation. The ELMO ontology uses a multi-layered architecture divided into highly maintainable, extendible, and reusable modules that can be used by organizations such as the Hellenic Transmission System Operator (HTSO). The ontology is primarily developed specifically for the electricity market of Greece, and the adaptation to other markets are thereby difficult.

Other examples are the studies of Santos et al. [3,47,48] that develop an Electricity Market Ontology (EMO). The EMO is an upper ontology for the electricity market, from which other low-level ontologies can be extended. It defines the main concepts of the electricity market, and the specific ontologies extended from the EMO define requests, responses, and notifications. Ontologies for the EPEX [3] and Nord Pool spot market [47] are developed as extensions of EMO. The research in [48] states that the aims of EMO are to be extendable and reusable in the development of other low-level ontologies for specific markets, such as MIBEL or IPEX (The Belgian and Dutch electricity market).

3.3.3. Ontology Design

Gruber provides five design principles [86] for the development of ontologies: clarity, coherence, extendibility, minimal encoding bias, and minimal ontological commitment. For ontology design, it is necessary to consider the ontology representation languages including tools to create and manage ontologies. Some standard ontology languages have been established with stable tools for the Semantic Web community, e.g., the Resource Description Framework (RDF) [87], RDF schema (RDFS) [88] and the Web Ontology Language (OWL) [88].

Several features of the Semantic Web languages are important for the ontology development, e.g., Open World Assumption (OWA), Description Logics (DL), and service representation. OWA assumes that knowledge is always incomplete. It is very important because incomplete information is common, and fragments of knowledge are often distributed within multiple ontologies [89]. Comparatively, the Closed World Assumption (CWA) assumes that if a statement cannot be proved to be true then it is false. DLs are formal languages designed for knowledge description and standard reasoning and provide the underlying formal framework for OWL and RDF [90]. DLs are known as the basis for ontology languages and are used to define, integrate, and maintain ontologies [89]. DLs are discussed in [24,48,91].

Together with the introduction of the ontology design. Semantic web services are an integral part of the Semantic Web and aim to be automatically discovered and invoked by computer programs [92]. Therefore, semantic web services must be able to describe the provided information and how this information can be retrieved [93]. A number of languages are available to describe services, e.g., OWL-S [94], Web Service Modeling Ontology (WSMO) [95], WSDL-S [96], and FLOW [97].

• Categories of ontologies

Ontologies can be categorized into three levels: upper ontologies, domain ontologies, and application ontology (shown in Figure 2) [98]. Upper ontologies provide common and consistent concepts that are referenced by other ontologies. Several upper ontologies exist, e.g., Suggested Upper Merged Ontology (SUMO) [99] and DOLCE [98]. Domain ontologies reuse or specialize concepts from the upper ontologies, and specify terms, relationships that are relevant in a particular domain. For instance, the domain ontology in [15] describes the concepts of the process dynamics, control, automation and the services provided by the agents, and defines relevant classes of entities and relations between entities. Application ontologies re-use and extend terms from one or more domain ontologies to apply for a specific application, and generally cannot be reused for other applications.

In the energy domain, ontologies for complex systems are often separated into a hierarchy consisting of an upper ontology that is connected to several lower-level ontologies representing specific subdomains [100]. The three MASs (MASCEM, ALBidS, and MASGriP) developed by Santos et al. [21,101,102]. Are all framed by an upper ontology, which allows communication between the simulations. However, this approach requires universal acceptance from all entities involved, and the low-level ontology for each layer still needs to be extended. Dam, Nikolic, and Lukszo [103] propose the generic ontology and the case-specific ontology, where the case-specific ontology is a specialization of the generic one and the generic ontology is a generalization of all underlying case-specific classes shown in Figure 3. Dam, Nikolic, and Lukszo [103] also suggest how to decide on the borders of the generic and domain-specific class in ontology. In [85], the ontology is divided into smaller building blocks, which makes it easier to modify and reuse in other models. [16] proposes a hierarchical ontology for the energy supply structure of buildings (shown in Figure 4). This proposed hierarchical ontology aims to investigate the interaction between energy flows on different aggregation levels within a building.

• Ontology mapping

MASs are usually developed independently and may not use the same upper ontologies. Therefore, ontology mapping is needed when an application requires access to multiple individually created ontologies. The mapping between ontologies can take much work [104], and there are several ontology mapping tools available [105]. Among the selected literature, ontology mapping is discussed and presented in [9,21,49,91].

• Ontology development tools

There are many tools for developing ontologies [106], e.g., Protégé (https://protege.stanford.edu/) and SWOOP [107]. Protege is well established and used by a large user community. For instance, Protege is used in the selected literature [23,24,48,50].

• Ontology development process

Ontology development processes is a relatively new field of study, including ontology life cycles, methods, and methodologies for building ontologies [89]. [108] introduces a methodology for ontology development including three phases: specification, conceptualization, and implementation. Noy and McGuinness [76] propose a more detail and practical ontology development process with seven steps which have been popularly used:

Step 1.

Determine the domain and scope of the ontology

Step 2.

Consider reusing existing ontologies

Step 3.

Enumerate important terms in the ontology

Step 4.

Define the classes and the class hierarchy

Step 5.

Define the properties of classes—slots

Step 6.

Define the facets of the slots

Step 7.

Create instances

3.4. MAS Design and Architectures

3.4.1. MAS Design Methodologies

According to [109], the MAS design usually consists of

(1)

A conceptualization phase where the problem to be solved is specified;

(2)

An analysis phase;

(3)

A design phase that uses the results of the analysis phase to produce agent designs of varying detail

Although the majority of the selected literature not specifically present their phases of the MAS design methodology, the introduction of the MAS architecture/structure in their cases is more or less according to the Gaia methodology (shown in Figure 5). The Gaia methodology is popularly adopted for the analysis and design of the agent-based system, it is used in [23,24,50]. Some other similar methodologies are also used for the agent-based system design, e.g., High-Level and Intermediate Models for Agent-oriented Methodology (HLIM), Modelling Agents and their environment (AUML), MASE [24].

Another MAS design methodology proposed by the IEEE PES MAS working group(http://sites.ieee.org/pes-mas/agent-technology/design/) is mentioned in [17]. This MAS design methodology is proposed by [110] with six stages, and each stage of the methodology produces material that is input to the next stage (shown in Figure 6):

• Requirements and knowledge capture stage: the MAS design usually begins with a particular problem. To solve this problem, this stage specifies the system requirements and capture the knowledge needed to fulfill those requirements. The system requirements and captured knowledge is the input to the next stage.
• Task decomposition stage: it transforms the requirements specification and captured knowledge from the previous stage into a hierarchy of tasks and subtasks. These tasks may include the functions performed by legacy systems.
• Ontology design
• Agent modeling stage: based on the task hierarchy and ontology design, it identifies a group of autonomous agents performing the tasks in the task hierarchy. Each task in the hierarchy must be attributed to at least one agent and one agent can encapsulate one or more tasks. The outcome is a set of agent models that specify the tasks the agents perform. The tasks attributed to legacy systems and generated new codes are also identified at this stage.
• Agent interaction modeling stage: it defines the interactions the identified agents support. The output usually is the interaction diagrams.
• Specification of agent behaviors stage: it specifies the interaction functionality of the agent and the control functionality of the agent.

Some MAS design also defines layers of the MAS architecture, e.g., [23] that the MAS system architecture includes two layers: the management layer and the subjacent execution layer. The management layer is responsible for functionalities that can be considered general in the frame and are covered by the agent control systems. The subjacent execution layer employs the automation agents’ tasks.

• MAS Development environment

The MAS development environment in the selected literature is usually performed using JADE (Java Agent Development Environment) [21], e.g., in [17,23,24]. JADE is one of the agent platforms compliant with FIPA standards. JADE provides services such as agent management system, directory facilitator, agent communication channel, etc., and supports the paradigm of avoiding SPOFs (Single Point of Failures). JADE has limited support for Semantic Web technologies. Therefore, some extensions are usually used to compensate for this issue. For instance, AgentOWL provides support for OWL ontologies using JADE agents [112] and AgentScape attempts to deal with scalability issues [113].

3.4.2. MAS Architectures

The MAS architectures (sometimes also called MAS structure, or MAS organizational structure) in the selected literature includes agent types and agent management framework/ system architecture, followed by the agent communication and ontology design that are usually introduced together, sometimes with agent interaction/activity diagrams.

• Agent types

The agent types are defined based on the system requirements, captured knowledge, and decomposed tasks. For example, agent types represent the devices and units in a power system, e.g., building management agent and RES (Renewable Energy Resources) agent [8], distributed voting agent and monitoring/resurrection agent [20], bus agent and switch agent [23]. In some MAS, the agents control the corresponding equipment according to their objectives, the measured and collected data, etc.

Agent types represent market players that especially for the electricity market, e.g., user agents and energy market control agent [17], system operator agent and VPP (Virtual Power Player) agents [37]. Some agent types are also ontology related agents, e.g., translator agent and ontology Agent [53]. In [53], the translator agent communicates with the main controller function blocks, and the ontology agent extracts knowledge from the ontology-based on requests.

Sometimes, the agent goals are also introduced together with the agent types. For instance, in [8], the agents act to achieve three goals of system load supplying, energy cost minimization, and residents’ comfort maintenance.

• Agent management framework/ system architecture

The structure of a MAS usually is illustrated in the agent management framework/ system architecture. For instance, the proposed agent management framework [8] including several components, e.g., the agent platform, agent container, and directory facilitator, etc. The system architecture also can visualize the multi-layered structure, e.g., [20], and the relations of agents and environment, e.g., [17], and relations of agents and physical systems, e.g., [23].

3.5. The Application of Ontology in MAS Development

3.5.1. MAS Interoperability and Ontology

In a MAS, it is important to set up a communication language for meaningful conversations between agents. The agents communicate through message exchange so-called Agent Communication Language (ACL). ACL is the existing interaction language standard for exchanging knowledge between agents. For a given Communication act «F(P)», the «F» part refers to the MAS and is regulated by the ACL standard, and the «P» refers to the domain knowledge. In our case, «P» refers to the «Energy Domain» or the «Energy Ontology» [114].

However, even an agent development environment supporting the same Agent Communication Language (ACL) and content language are implemented in two MASs, it does not mean that the agents in the two MASs can share any useful information because different ontologies are used in the two MASs [91].

As more applications of MAS in the energy domain for advanced functions and MASs are not expected to operate in isolation from each other, the interoperability challenge raises due to an increasing requirement for data and information exchange between systems. Therefore, there is a need for full interoperability and open standards for the MASs in the energy domain [91]. The interoperation issues of existing multi-agent systems have highlighted in the literature, particularly the issues of the use of different ontologies. Meanwhile, it is important to establish the same language, especially a common ontology for the communication between agents.

3.5.2. Agent Communication and Ontology

Agent communication in MAS can be accomplished in two ways: immediate communication among agents and interaction in a unitive environment [13]. MAS usually implements higher-level communication and supports reasoning abilities based on the Agent Communication Language (ACL) and a common vocabulary defined in an ontology [115]. The agent communication and ontology design in the selected literature is similar to the combined stages of ontology design, agent modeling, agent interaction, and specification of agent behaviors stages proposed in [110], and usually consist of standards for agent communication, interoperability, and ontology design.

• Standards for agent communication and interoperability

A standard for the communication between agents has been proposed by the Foundation for the Intelligent Physical Agent (FIPA [116]. The FIPA standards have been popularly used by MAS developers in the computer science community and FIPA was formally accepted as a standards committee of the IEEE Computer Society In 2005 [109]. Such standardization promotes open specifications for the interoperability between agents and MAS [117]. The FIPA standards include specifications for the agent communication language, communicative acts, content languages, and message transport protocols. It also includes a standard that proscribes the agents that a MAS must implement to be FIPA compliant

The FIPA-ACL specifies the syntax, the content of the message provides the semantics of the message including the content language and the ontology [118]. The messages built under the ACL structure allow the definition of various elements (e.g., performative, sender, receiver, content, language, and ontology, among others) and various communicative acts (e.g., agree, cancel, confirm, not-understood, etc.) [9]. Meanwhile, the correct interpretation of the meaning of the message is assured, the ambiguity is removed about the content [21]. The MASs in [3,47,48,52] all apply the FIPA-ACL. There are other ACL investigated in the literature as well [119,120], e.g., Open Agent Architecture (OAA) in the work of Praca et al. [42].

MASs developed by different platforms can interoperate with these FIPA standards, but it doesn’t mean that useful information can be shared between agents if the MASs employ different ontologies [21,91]. It requires MASs share a common vocabulary, so the messages may be interpreted correctly among agents [47]. Therefore, ontologies are used to enabling the standardization of communications and interpretation of concepts between MASs [48].

The IEEE standards committee has identified the challenge of interoperable protocols, data formats and meaning and stated that open communication between smart devices using common protocols is crucial to interoperability [121]. Some standards in the power systems promote interoperability between devices within substations and open interfaces between energy management systems [91,109]. The most widely applied standard in the power system is the IEC 61970 Common Information Model (CIM), and its distribution management extension IEC 61968 [122].

IEC 61970 Standard is proposed by the International Electrotechnical Commission (IEC) to discuss and plan a variety of electrician and electron standards in order to procure international cooperation. IEC 61970 Standard defines the application program interface (API) of the energy management system is promulgated by IEC No.57 technical commission (Group 13) [13]. There are five main parts in the IEC 61970 standard: introduction and basic request, glossary, common information model (CIM), and two levels of component interface specification (CIS).

The CIM is a three-layer domain model, it defines a common vocabulary to describe the basic components used in electricity transportation and distribution [38], and CIM aims to facilitate power management processes (e.g., outage management, asset management, and customer information management) [50].

To achieve coherent and advantageous cooperation between different power systems, some reference models and frameworks are also popular used, e.g., SGAM (https://sgam-toolbox.org/) (the Smart Grid Architectural Model), USEF (https://www.usef.energy/) (the Universal Smart Energy Framework), and SEAS knowledge model (https://www.the-smart-energy.com/) (Smart Energy Aware Systems).

The Open Automated Demand Response (OpenADR) (https://www.openadr.org/) and energy@home (http://www.energy-home.it/SitePages/Home.aspx) models are also highly discussed in the literature. However, [50] states that ‘none of these standards cover the whole semantics involved in a flexible urban energy network on its own, and they are not formally aligned with each other’. For example, the term ‘equipment’ could refer to transmission system equipment, or domestic appliance equipment [50].

• Ontology-based agent communication design

According to FIPA [107], semantic MAS interaction can be specified with three dimensions: 1) Internal agent behavior: action selection and execution; 2) External (agent) interaction to exchange: a) content of the interaction including both information and tasks; b) context of the Interaction and its relation to an agent organization; 3) System, or platform, services: message transport, discovery, action execution, management, and inter-platform interaction.

The FIPA (agent interaction) model (often referred to as the FIPA-ACL) is an Agent Interaction Protocol Suite (AIPS). The AIPS contains several distinct semantic protocols for agent communication including interaction process, communicative acts, content logic, and content ontologies (shown in Figure 7) [107].

The design of internal agent behavior and interaction in a MAS mainly concerns the agent communication models as in the majority of the selected literature. The design of agent communication usually includes messages (message content) and message exchange (protocol). Messages and protocol are usually described in the UML diagrams as class diagrams and sequence diagrams, e.g., in [20], the communication sequence and communication parameters are introduced. The content of a message comprises two parts: content language (provides the syntax or grammar of the content) and ontology (consists of the semantics or lexicon of a message) [91]. The ontology-based agent communication model can be shown in Figure 8.

MAS developers usually use JADE to create agents because JADE agents communicate by exchanging message in compliance with the FIPA ACL. The FIPA Semantic Language (FIPA-SL) is popularly adopted as the standard content language [123]. In FIPA-SL, an ontology comprises a list of concepts, predicates, and actions specific to the domain of communication. However, the structures of ontologies in the selected literature are different. For instance, the ontology in [124] is defined in the form of EBNF and includes seven parts (policy, modality, trigger, subject, behavior, target, and constraint), and the ontology in [20] contains four parts (ID, type, parameter, and value).

When designing a MAS, developers usually introduce the syntax and semantics of the domain ontologies and application-specific ontologies applied in the MAS and describe the purposes and functions of the ontologies. For instance, [124] applies a policy ontology in their MAS. In [124], the policy ontology regulates behaviors of agents including application activity, authorization activity, monitoring activity, requesting-monitoring activity, discovery activity, and negotiation activity. This research designs a policy engine within each agent who is the subject of obligation policies or the target of authorization policies and the policy engine interprets and enforces the policy when the policy is enabled.

The FIPA agent standards focus on specifying protocols for external interaction and platform services rather than on the internal agent behavior [111]. It is because the internal agent behavior is are often problem-specific or application specific, and not easily accessible and observable. In the FIPA Ontology Service, an ontology agent is recommended to provide a number of ontology-related services for solving the problem of using multiple ontologies [91].

However, this solution is difficult to be implemented due to challenges of the system integration including between-ontology mapping, translation mappings, etc. Therefore, [91] recommends defining a common upper ontology that represents the general concepts used in the domain of power system. Meanwhile, related common standards in a domain can serve as a foundation for an upper ontology, e.g., The power systems Common Information Model (CIM) [126]. The upper ontology for the MAS interoperability of the electricity markets and demand side is well discussed by Santos et al. [3,47,48].

4. Discussion

The literature shows that there is an increase in MAS application in energy domain since the distributed nature of MAS allows the energy system design to deal with complex systems [127]. In a MAS, complete knowledge about the system is not required, but each agent in the system acts autonomously toward some predefined objectives to optimize the system performance [128]. Therefore, agents have possibilities to represent different market participants, network components, or systems [9]. The agents’ individual goals decide the agents’ behaviors, e.g., either cooperate or compete with other agents [127]. The behavior of the overall system is a result of the agents’ behaviors.

MAS is not necessarily a simulation tool, but simulations may be important for the study of the energy domain, e.g., scenario comparisons, evolution studies, and sensitivity analyses. Several MAS studies are found in the literature dedicated to the energy domain. For instance, the study of Koritarov [1] demonstrates the application of the EMCAS in electricity markets. The model enables the investigation of the physical infrastructure and the economic behaviors of the market participants. The study of Li et al. [39] demonstrates the AMES simulation for the wholesale operations and market participates strategies. In the building sector, the simulations of electricity consumption in an office building are simulated by Mousavi et al. [53]. The study considers the unpredictable nature of business processes. Meanwhile, the research by Zeiler and Boxem [16] simulate the grid conditions in their study of building control.

All these simulations aim to solve problems in specific domains and are limited to an existing system (do not allow for connections to external systems) or do not take advantage of the formal exchange of knowledge. It is possible to solve problems that cover more complex domains if these systems can communicate and exchange knowledge with each other.

The combination of different systems can simulate a complex system such as the energy system. In such a system, stakeholders work together, interact, and negotiate with each other, while the demand and supply of resources need to be managed. The heterogeneity among these systems makes the interoperability complex, and the system may have different domains, concepts definitions, programming languages, etc. In order for the MAS to be able to communicate with each other and overcome their individual limitations, a mechanism for communication is important. This mechanism should allow information and knowledge sharing. At the same time, the system should be flexible to deal with several processes. Therefore, a communication standard should be defined, ensuring that agents in the system use terms with the same meanings [129].

The FIPA is the de facto standard for agent development [9]. FIPA provides different interoperability standards, e.g., the standard agent communication language (FIPA-ACL), which make it possible to integrate different MASs [130]. However, it does not mean that agents belonging to different MASs can share any useful information if the MASs use different ontologies. The ACL provides a framework for the communication standardization between agents, but the standard only defines the structure of messages and interactions. Therefore, agents speak the same language but do not share the same vocabulary.

In an ACL, the content of messages must be understood by agents for the messages to be meaningful. Catterson et al. [91] describe it as “…the structure and meaning of the content are in a format expected by the receiving agent so it can decode the sender’s intentions”. Agents exchange information to achieve their goals and therefore must apply the same language to interact with each other. But it also needs a common representation of concepts for agents, which ontology can provide.

The ontology describes the concepts and the relations among agents and therefore must be a part of each agents’ knowledge base [131]. Ontology is described as a form of knowledge representation of the world or some parts of it and “provides a shared vocabulary, which can be used to model a domain that is, the type of objects, and/or concepts that exist, and their properties and relations” [132]. Meanwhile, Luncean et al. [131] states that “An ontology is used to represent knowledge that is shared between different entities. It provides terms and vocabulary used to represent knowledge so that both sender and receiver can understand” Several ontologies already exist in the energy field. In [16,52], the main goal of ontologies is to support the interactions between energy management of buildings and the smart grid.

It is important to mention that the design of an ontology itself does not contribute to energy savings or energy-neutral building environments. However, it brings several benefits to the design of the software process of a MAS. First, it gives a deeper insight into the modeled domain and system functionality. Secondly, it reflects upon the data types and required communication between agents. These factors are useful when concepts are shared between different teams and systems, e.g., when different domains need to be connected to a smart grid.

However, MASs in the energy domain are developed with their own ontology, which cannot be directly integrated into other systems. A standard to solve the problem of multiple ontologies would lower the cost and human effort when different systems need to be connected. In the literature [91], several solutions for MAS integration are investigated. The FIPA ontology services the integration of existing MASs by introducing an ontology agent. This agent provides ontology-related services, e.g., translating expressions between ontologies and identifying a common ontology to two agents [16]. However, ontology designers still need to identify the similarities and differences between ontologies manually to translate the ontologies. This likely introduces more complexity and potential errors.

An upper ontology, as discussed in [21,103] could be an alternative to represent the general concepts of the domain. Such ontologies provide the framework in which the low-level ontologies can work. The upper ontology allows communication between different systems and each system with separated low-level ontologies. An upper ontology can be defined through multi-layered architecture or smaller reusable modules. The development and maintenance of MAS are easier and more efficient by composing a large-scale ontology out of smaller ones. This makes the ontologies simpler to modify, e.g., if legislation changes. The independent parts of an ontology must be well defined and separated. Thus, it is possible to reuse the parts in similar applications. The layered architecture also makes the ontology easier to be extended for other application domains and not just the intended domain [91].

An upper ontology for the energy sector can serve as an open standard that can assist the development of multi-agent solutions. It should not be a standard for all applications, but a tool from which the low-level ontologies can be extracted. Upper ontologies for the electricity domain are found in the literature, but the integration with the entire energy sector is still missing. This integration is necessary to fully understand and control the energy sector because the energy sector becomes more complex and consists of multiple hybrid systems.

The literature reviewed in this study presents different energy domains and includes different agents, data, and terms. This heterogeneity hinders the full adoption of these MASs and ontologies in a real scenario. Hence, there is a need for developing a unified ontology that represents all energy domains and provides a common terminology. In the literature, business models are separated from the MASs in the energy domain. For a deeper understanding of the domain and related agents, business models should be considered as part of MASs.

The combination of MASs, ontologies, and business models will enable simulations of the energy sector for exploring the interplay of policy, economy, and technology. Furthermore, a standardization of communication between agent will provide better knowledge- and data exchange between agent and domains. However, better simulation tools which can be used for scenario comparison, prediction of future evolution and sensitivity analysis are important, and it will make simulations easier to predict future events, identify unmet needs and act deliberated to changes in the energy sector.

5. Conclusions

This study contributes a scoping review of literature on the application of ontology in the MAS for the energy domain. It is evident from the literature highlighted in this study that multi-agent ontology approaches are of emerging interests in the energy sector and that complex system modeling is an essential tool in assessing control strategies and new policies for designing more efficient systems.

The selected publications show that the application of ontologies in the field of MAS for the energy domain was mainly conducted after the year 2004, focuses on the sub-domain of grid control between 2004 to 2014, and mushrooms into the sub-domain of electricity market since 2014. The discussion of ontology and MAS in the selected publications can be divided into five categories:

• Definition of agent and MAS. The definitions of agent, intelligent agent and MAS and the introduction of an agent structure are given in some selected publication. However, some publications do not differentiate the agent-based system and multi-agent-based systems.
• MAS applied energy domains. The applied energy domains include grid control (also, microgrids), electricity markets, demand-side and building systems. The applied MAS tools are also introduced in some selected publication.
• Defined ontologies in the energy domain. Definition of ontology, functions of ontology and the defined ontologies in the energy domain are introduced. The ontology design is introduced usually together with the agent communication model. Although generic ontology and the case-specific ontology, upper-level, and lower-level ontology, and ontology hierarchical are introduced, a systematic discussion on the categories of ontologies (upper ontologies, domain ontologies, and application ontology) is missing. Meanwhile, although ontology mapping for inter-MAS communication and ontology development tools are introduced, the ontology development process is not yet discussed in the selected literature.
• MAS Design and architectures. The MAS design methodology-Gaia methodology is introduced and applied in some selected publication, and MAS design methodology proposed by the IEEE PES MAS working group is introduced but not well discussed or applied in the selected publication. The MAS Development environment, JADE, and its extensions are introduced but the design detail with JADE is missing. The MAS architecture is commonly introduced with the description of agent types and agent management framework/ system architecture.
• Ontology in the MAS development. The importance of ontology for the MAS interoperability is emphasized and the application of ontology in the agent communication design is well discussed in the majority of the selected publication. The standards for agent communication and interoperability are discussed with two dimensions: standards for domain-specific, e.g., the SGAM reference model, the power systems CIM and SEAS knowledge model in the energy domain are discussed; The FIPA-ACL is applied for almost all MAS design in the selected publication.

5.1. Recommendation of the Ontology-Driven MAS Development for the Energy Domain

Based on the review result, this paper finds out the following aspects in the ontology-driven MAS development for the energy domain should be further discussed, developed or emphasized:

• The ontology development process in MAS design

Although the importance of ontology in the energy domain has been emphasized, especially for the MAS interoperability. However, from the ontology engineering perspective, the ontology development process has not been addressed well in the MAS design, especially with the consideration of the ontology categories. This paper recommends the further work can combine the categories of ontology [98] and the ontology development process [76] into the MAS design with two aspects: multi-agent communication and MAS interoperability.

• The detail design process and realization of the ontology-driven MAS development

The selected publications well discuss the ‘what’ and ‘why’ of their designed/developed MASs. However, the ‘how’ is missing in the majority of the selected publication. Therefore, it is difficult for readers to re-produce their methodologies of MAS development. Therefore, this paper recommends the further work can focus on this aspect, and it is especially important for the MAS interoperability.

• Open standard implementation and adoption

Open standards for both MAS design, agent communication and energy domain are discussed in the selected publication, especially regarding the MAS interoperability. This paper finds that the MAS interoperability issue is not solely due to the inter-MAS communication barriers, and upper ontology design cannot solve this issue if the designed upper ontology or the selected open standard is not adopted by other MASs. Therefore, further work on the open standard implementation and adoption for the ontology-driven MAS development is recommended.

• Higher intelligent MAS development

The MAS interoperability is important for the distributed energy systems, and ontology improvement (upper ontology or generic ontology) seems like the only solution in the majority of the selected publications. This paper recommends the future work can consider developing higher intelligent MASs that allow the ‘fuzzy communication’ between MASs.

• Inter-domain MAS development

Although this paper tries to search literature in the energy domain for both electricity and heating. However, the search result only shows in the electricity domain, and the literature on MAS and ontologies for the heating sector is missing. Heating is an important subdomain in the energy sector and is also strongly connected to the electricity sector through combined heat and power generation, and electrical heating. Hence, heating should be equally addressed in the studies of MAS and ontologies for the entire energy sector. The priority for future work in this field should focus on the interoperability with further external systems and cover the simulation of other areas in the energy system, including heating. However, the inter-domain ontology design will be more complex and difficult compared to only under the electricity-related domain.

• Agent listing

Agent types, roles, and interactions are well introduced in the selected publication. Meanwhile, the domain analysis in the MAS design methodologies is introduced. Some studies have done illustrations of agents in smaller scales, e.g., [3,47,48,52]. However, a systematic approach to list all related agents with a clear MAS boundary is missing. In a MAS, agents are specialized to perform tasks based on their individual goals [133]. Meanwhile, a MAS with stakeholder listing can give a good overview of the whole system. The literature shows that there are different ways to illustrate the identified agents together with their relationships. Some authors [53,81] introduce agents with descriptions, and others [18,52] use diagrams to graphically present agents. One example of the graphical illustration is the Harmonised Electricity Market Role Model by ENTSO-E [134]. This Harmonised Electricity Market Role Model represents agents, their roles, and information flow between them. This role model provides a common definition of roles and domains employed in the electricity market. It enables a common language in the development of information interchange.

Another way to present and describe stakeholders is by using business models. The research by Xia et al. [135] investigates the Swedish mobile phone business ecosystem. The stakeholder listing is represented by the Osterwalder and Pigneur business model canvas. An overview of the agents, their interrelations, and information flows can be illustrated in the business model canvas. The homogenous setup provided by the business model canvas highlights and organizes the identified information. This simplifies the information search. Furthermore, the business model canvas can easily be extended with new stakeholders by following the canvas approach. Both stakeholder listing by diagrams and the business model canvas provide well-organized information about complex systems. The canvas approach makes it possible to include supplementary information about the stakeholders.

5.2. Limitations and Future Work

This paper applies a transparent scoping review methodology through the entire process. To ensure a broad search of the literature, the search strategy includes four online databases, resulting in over 1400 articles. However, the search result may still not identify all relevant articles in the literature despite this paper attempts to be as comprehensive as possible. Ontology is a recently established word in information science [71], therefore, an extension of the literature search including the terms “domain knowledge” and “knowledge representation” may result in additional literature in the field of MAS and ontology.

Furthermore, the fields of organizational theory and business ecosystem are not included in this paper because the literature search only focuses on the energy domain. The energy domain consists of multiple agents and can be considered as an ecosystem in which a community of organisms interacts with each other and the surrounding inorganic environment. This biological definition of an ecosystem is first introduced in [136] and is later adopted in the business domain [137,138,139,140,141]. A business ecosystem is a network of players that are bound together through collective activities to produce an entity that offers value for customers and meet their requirements. MAS in the smart energy domain is similar to this since all types of stakeholders, e.g., electricity traders, building managers, and commercial heat pump providers are connected and interact with each other to offer value for the entire system. Meanwhile, the economic globalization, increasing number of transnational organizations, and rapidly information technology changes increase the complexity of the energy domain, and computational models provide opportunities to understand and respond to these changes [142]. Therefore, a review of the organizational research and business ecosystem in the MAS-orientated energy domain should be considered for future work.