Evaluation Method for Autonomous Decision-Making Performance in Energy and Environmental Innovations: A Case Study of an Indonesian Community

Abstract

This paper develops an evaluation method for assessing autonomous decision-making performance and demonstrates it using a case study. Focusing on community decision-making practice in energy-environmental innovation projects, a decision-making model is developed using Petri-net. This empirical model is then expanded to be able to accommodate autonomous properties and more pathways to reach various decision-making outcomes. The autonomous decision-making performance evaluation is employed by simulating the impact of various levels of autonomous conditions using the expanded model stochastically. Those results are further divided into six categories, based on the conditions (autonomous, semi-autonomous, and non-autonomous) and decision outcomes (fully successful, moderately successful, and failed). For each category, the specific stakeholders’ properties are analysed and explained. The categorised conditions are useful for estimating the outcomes of the particular community decision-making practice based on the stakeholders’ properties. The model can be modified in order to pre-evaluate other energy and environmental related decision-making.

1. Introduction

The continuous global pressure to improve the environmental sustainability of the energy sector has been responded to with technological advances and increasing use of renewable energy. World renewable energy production continues to show an increasing trend in both absolute and proportional terms [1]. The increase of renewable energy utilization is creating a gradual shift in the structure of conventional energy systems from centralised to decentralised systems. This situation presents an opportunity for alternative energy systems to become more distributed and localised.

Distributed energy systems, by nature, tend to be more localised. Renewable sources such as biomass, wind, geothermal, solar, and hydro, are subject to local availability, and the dispersed nature of these energy sources means that they are commonly considered to be deployable as a distributed energy system. A new decision-making approach is then necessary to prepare for the emerging trend of a more decentralised energy system. Distributed, localised energy systems not only require a wide range of stakeholders to collaborate, but further call on the stakeholders to make decisions by and for themselves. From the demand side, reliability issues which may cause challenges with renewable energy systems are widely considered solvable by shifting to smart grid [2]. From the social acceptance point of view, smart grid applications for distributed energy systems may be enhanced when the autonomy of end-users is strengthened through policy rather than by the tightening of regulations on individual consumption [3,4]. This point of view puts a new emphasis on the concept of local public engagement in energy decision-making, and makes way for the practice of making decisions autonomously by all actors, not only for consumers or individual end-users, but including formal institutions such as governments, organisations in the private sector, and semi-formal institutions such as community groups.

This study starts with the premise, supported by earlier research [5], that the ability of stakeholders to think and act autonomously is an important condition for decision-making and implementation, especially in the emerging era of distributed and localised energy systems. Whilst many studies have been undertaken focusing on technological aspects of renewable energy deployment via decentralised systems, studies in understanding the social aspects of energy and environmental innovations are predominately focusing on consumer behaviour for demand side management [3], or even the more recently coined term demand side participation [6,7]. However, some studies have stated that these methods, such as smart-metering, are not likely to be sufficient if the era of renewable energy deployment via smart grids is realised [4,8]. Renewable energy initiatives that have been autonomously performed by national governments, or city governments, or private business entrepreneurs, have shown surprisingly high rates of accomplishment compared to strategies that were integrated under international approaches or the traditional “political-energy business” complex [9]. The fact that renewable energy deployment through distributed systems is becoming closer to realisation, has set the expectation that decision-makers must prepare for this by becoming more autonomous or encouraging greater autonomy. Autonomous decision-making in energy and environmental innovations is therefore proposed as an important decision-making approach for responding to the recent shift in energy system structure. The aim of this paper is to develop and demonstrate a method to evaluate the performance of autonomous decision-making in energy and environmental innovations. In certain conditions and cases, autonomous decision-making is more suitable and leads to successful decision implementation. In other cases, there is also a possibility that making decisions autonomously does not lead to better outcomes compared to other approaches. Although it is known normatively that stakeholders’ autonomy in making decisions is important, its role and contribution in decision-making has not been objectively examined [10]. The fact that no studies could be identified that objectively investigate the role of autonomy in decision-making on energy-environmental innovations shows that this theme has to-date been insufficiently explored. There is a need to develop a method that objectively evaluates autonomous decision-making performance and can demonstrate the relative effectiveness of a decision-making system with autonomy for making and achieving decisions, particularly in the energy and environmental field.

In order to structure the evaluation of autonomous decision-making performance, an evaluation method is developed. Three basic steps need to be completed before the evaluation is performed. First is the analysis of the decision-making process, which was done by taking five case studies of community decision-making processes in the energy-environmental sector. Among twenty communities that initially observed, these five cases were selected because they have performed project related with energy or environmental innovations and have sufficient information regarding the history and processes. This analysis involved two procedures of decomposition and extraction, and resulted in general decision stages and stakeholder autonomous properties. Secondly, the generalisation and modelling of autonomous decision-making was undertaken. The analysis results were applied back to one of the case studies in order to generalise and expand the practical Petri-net model. This model was designed to be able to accommodate various changes of the stakeholders’ properties, and also contains several critical points that are used in assessing the evaluation results.

The third step, which is the focus of this paper, is employing a stochastic simulation to the model for evaluating the performance of autonomous decision-making systems. Various combinations of properties are simulated randomly to understand whether autonomous conditions will yield a successful decision-making outcome or not. The simulation results are then evaluated using performance indicators to understand the performance of autonomous properties in the decision-making process. Through this method, we are able to identify which conditions can lead to successful decision-making. The overall framework for evaluating autonomous decision-making performance is presented in Figure 1.

2. Autonomous Decision-Making Definition, Properties, and Model from Case Study

This section comprises a brief explanation on the autonomous decision-making concept and definition. Autonomous decision-making properties that were extracted from the case studies using the developed method are also explained here. A case study is then selected from among the five cases and an autonomous decision-making model constructed for the particular case in the form of a Petri-net. This section provides three important steps performed to further examine and model the autonomous behaviour in decision-making process: (1) decomposing the community decision-making and stakeholders’ properties; (2) extraction of stakeholders’ autonomous properties; and (3) developing an autonomous decision-making model. These procedures can be replicated on other case studies.

2.1. Autonomous Decision-Making Definition and Properties

Autonomous decision-making in this study is defined as a process where the decision-makers have the ability to recognise a problem or select a goal, and make decisions towards solving the problem or achieving the goal of their own volition or based on their own perspectives. Autonomy, was originally defined as the property of a state or city to administer their own affairs [11]. This means that autonomy is a property owned by a certain state, or territory, or government institution. It was not until the Renaissance era that autonomy was applied as a term for individuals, through the work of philosophers such as Jean-Jacques Rousseau and Immanuel Kant [10,12]. The Kantian concept of autonomy was then developed and “individualised” further into “personal autonomy”. The concept of personal autonomy is the cultivation of one’s individual self, focusing on whether a person is acting for his or her own reasons [13]. Today, autonomy is understood in many different ways, depending on which field of study it is being viewed from. From the various concepts of autonomy aforementioned, one study defined the dimensions of autonomy as: (1) self-determination; (2) self-governance; and (3) self-authorization [14]. Another study focused on self-directedness and resoluteness dimensions of autonomy [11]. Meanwhile other fields of knowledge, such as computer science and information technology, view the ability to continuously learn or the “self-learning” trait in the emergence of autonomous machines or artificial intelligence as one of the most important characters of autonomy [15].

This study views autonomy as a property of a person as a decision-maker, or as a stakeholder who involved in the decision-making process. This does not mean that the decision-maker is acting individually, but that they can autonomously decide to interact, share and collect information, and collaborate with other stakeholders to achieve a certain goal. Each decision-making process contains stakeholders’ properties, which influence the outcomes of the decision. In an autonomous decision-making process, the stakeholders’ properties exhibit autonomous behaviour, and therefore are further labelled as stakeholders’ autonomous decision-making properties.

The identification of stakeholders’ autonomous properties is performed via two procedures: decomposition and extraction. We have developed a method to carry out these two main procedures. This method, simply called the decision-making decomposition method [16], was applied to five case studies of community decision-making in energy-environmental projects.

The first procedure carried out was decomposing the community decision-making into decision-making stages. The application of this method to the five case studies yielded a common pattern of decision-making stages as follows:

(1)

Problem definition

(2)

Formulation of alternative solutions

(3)

Agreement or consensus building

(4)

Decision implementation (in many cases, this is the physical construction of a facility)

(5)

Management (Operation and Maintenance period)

(6)

Termination of the project (budget-wise)

(7)

of the project (in some cases)

The second procedure is the extraction of the stakeholders’ role or stakeholders’ general properties from each stage. This procedure has resulted in a list of twenty stakeholders’ properties that generally exist in the community decision-making process.

The identification of the stakeholders’ autonomous properties in decision-making is performed by cross-correlating the list of stakeholders’ general properties with autonomous behaviour elements such as self-determination [9,17], self-control [18,19], self-governance [3], and self-learning [15]. This procedure has resulted in seven stakeholders’ autonomous properties classified as follows [5]:

(1)

Motivation, initiative.

(2)

Leadership, ability to organise.

(3)

Self-learning, ability to manage information.

(4)

Interaction between community members.

(5)

Networking and collaboration with other stakeholders.

(6)

Persuasion and negotiation ability.

(7)

Responsibility and commitment to implement the decision.

Various combinations of these properties form the decision-making conditions. These conditions are the primary input for evaluating the performance of the autonomous decision-making. Further explanation on the decision-making conditions is presented in Section 3.

2.2. Generalisation of the Case Study and the Expanded Model

This section highlights the generalisation of the decision-making model from the practical case study into an autonomous decision-making model. Here we took one case study of a community decision-making process and built the expanded empirical Petri-net model based on the actual case study’s documented process. This empirical model was then generalised and expanded into an autonomous decision-making model, which was constructed by internalising the results from the decomposition and extraction procedures. The result is a Petri-net model for the autonomous decision-making process, which generally has fewer stages than the real-world process, but has the ability to be expanded to accommodate changes of stakeholders’ properties. The latter factor is the most important aspect of this expanded Petri-net model, because the stakeholders’ properties can then be adjusted and the results can be simulated to understand the performance of autonomous decision-making.

In this study, the decision-making process is regarded as a discrete event system. Petri-net is selected as a tool to model the decision-making processes because it can model complex interactions between the stakeholders, and can perform simulations to evaluate autonomous decision-making performance.

The case study selected is a community decision-making process for managing solid waste and transforming it into fertiliser and biogas through composting and installation of a bio-digester. The uniqueness of this case study lies in the application of two separate techniques of waste management system (WMS), which are a composting centre and bio-digester installation. The bio-digester installation project was an improvement to an existing composting project in the Rukun Warga (RW) 11 community in Bandung City, Indonesia. The RW 11 community is one of the low-income slum areas in Bandung, inhabited by 800 households in a dense urban area. In 1996, this community was selected by the Ministry of Public Works’ Bandung Branch to be a pilot project for community-based sanitation improvement, as a project under the Community-based Basic Infrastructure Improvement Program (CBIIP). With the assistance of Bandung City government, the community was assisted to decide upon and construct a composting centre located in RW 11 to improve the poor sanitation and waste situation [20]. The composting centre was successfully constructed, and the waste from within and around RW 11 was reduced. However, the operation became stagnant after the project ended.

A few years later, the composting centre was then replaced by a bio-digester installation. After the budget termination, the composting centre was still run by the existing Community Based Organisation (CBO). However, with no further financial help from the government, production could not sustain itself, and therefore the operation was terminated 15 years after the project started. Since the community’s paradigm on waste had changed, they still maintained the waste segregation activities. The segregated waste was then sold or reused by the women’s organization (My Darling) for making handicrafts. In addition to this, the existing CBO tried to seek financial support by submitting proposals to international and national non-governmental organizations (NGOs) [21]. Eventually, with assistance and consultation from academic scholars, the Environmental Agency and a local NGO, and financial help from the local bank, the composting system was changed to a bio-methane system, which produces biogas for households and liquid fertilizer. A recent study investigated the economic and social impacts of the biogas system, and found that the system is not economically feasible but it is socially accepted by the community [22]. The practical model of this case study is represented as a Petri-net in Figure 2 and the legend of places and transitions is in

Table 1. Legend for practical model of Rukun Warga (RW) 11 decision-making process.

Place	Description	Transition	Description
P1:	Waste and sanitation problem situation	T1:	Information about waste problem
P2:	Motivated national government	T2:	Problem recognition and planning for CBIIP
P3:	Motivated local government	T3:	Socialisation and persuasion
P4:	Selected community	T4:	Community gathering
P5:	Acceptation	T5:	Project executed
P6:	Community is motivated, the RW 11 Forum is established	T6:	Composting training for community members
P7:	Composting centre is constructed	T7:	Finance support is finished
P8:	Compost production, waste reduction occurred	T8:	Motivation to continue project is high
P9:	Composting project is stagnant	T9:	Offer from experts and NGO for bio-methane system
P10:	Proposal submission and expert consultation	T10:	Collaboration
P11:	Community gathering	T11:	Operation and Maintenance continued
P12:	Bio-digester system is installed
P13:	Community members become empowered and change of behaviour occurred

.

The practical Petri-net from the observed case study is then expanded into another Petri-net model that can accommodate more paths and results to enable a simulation that includes alternative decision outcomes. This Petri-net is hierarchical and includes sub-layers. The purpose of these sub-layers is for use in assigning the stakeholders’ properties at the beginning of the simulation. Thus, the number of stakeholders and the properties can be modified easily. This has enabled the creation of many possible alternative paths. The expanded model, which is later referred to as the autonomous decision-making model, is shown in Figure 3 and its legend in

Table 2. Legend for the autonomous decision-making model of waste management system (WMS).

Place	Description	Transition	Description
P1:	Waste and sanitation problem situation	T1:	Motivation subnet
P2:	Set of stakeholders’ motivation level	T2:	Information subnet
P3:	Set of stakeholders’ ability to manage information level	T3:	Leadership subnet
P4:	Set of stakeholders’ leadership level	T4:	Problem finding process
P5:	Problem is defined	T5:	Designing alternatives process
P6:	Alternatives are designed	T6:	Decision-making
P7:	WMS technique is selected (decision is made)	T7:	Construction of WMS
P8:	WMS is constructed (decision is implemented)	T8:	Operation and Maintenance
P9:	Waste is reduced	T9:	Termination of the project
P10:	Project stopped
P11:	Project continued

.

Analysis of the structure of the autonomous decision-making model resulted in the identification of critical points of the model [5]. A critical point is defined as a certain transition in the model that depends on the conditions assigned at the beginning of the net. There is a possibility of a decision-making process ending at one of the critical points before reaching the end of the net. Analysis of the model in this study suggests that the critical points are:

(1)

Problem finding process (T4)

(2)

Designing alternatives (T5)

(3)

Decision produced or consensus building (T6)

(4)

Operation and maintenance (T8)

(5)

Termination of project (T9)

These critical points are used for determining at which stage of the decision-making process a simulation ends, and to obtain the outcome of the decision-making process. The critical points are mainly utilised in establishing the evaluation performance indicators (Section 3.1). The firing of transitions in the expanded Petri-net model in Figure 3 depends on the existence of certain combinations of the stakeholders’ autonomous properties. In general, the rules are as follows:

(1)

T4 is fired when there is a combination between the “motivation” property and “ability to manage information” property belonging to any stakeholder.

(2)

T5 is fired when there is sharing of information, this means that there are at least two stakeholders have the “ability to manage information” property.

(3)

T6 is fired when there is the “leadership” property, in a certain combination of stakeholders.

(4)

T7 is fired when there is the “leadership” property. This transition is fired when there is a token in P6.

(5)

T8 firing required three properties altogether. However, a different combination of the three properties in each stakeholder might yield different output.

(6)

T9 firing is automatic, however the combinations of the stakeholders property will determine the places where the token will be produced, whether it will end in P10 or in P11.

3. Methodology for Evaluating Autonomous Decision-Making

This section discusses the methodology developed for evaluating the performance of autonomous decision-making. The first section explains the elements simulated in the model and is for evaluation of autonomous decision-making performance. The second section presents the Petri-net model and method to apply the simulation to the case study.

3.1. Elements and Indicators for Evaluating Autonomous Decision-Making Performance

This section discusses the main elements of the model, which are the identification of stakeholders involved and the combination of the stakeholders’ properties. The explanation is then followed by the performance indicators for evaluating the simulation results. The first element is the stakeholder involvement in the decision-making process. The second subsection seeks to explain the various conditions that can possibly occur in the decision-making process. These conditions are the combination of the stakeholders’ behaviour/properties in each decision-making stage. The third subsection presents the decision-making indicators for assessing the simulation results. These requirements are included in the autonomous decision-making performance evaluation method.

3.1.1. Stakeholders’ Role Identification Based on the Case Study

The number of stakeholders simulated in the model may vary, depending on the observed case study. As shown in Figure 3, the expanded model includes three subnets where the autonomous properties were assigned to each stakeholder involved in the decision-making process. The expanded model can accommodate various numbers of stakeholders. As more stakeholders are involved, the number of possible conditions will increase exponentially, hence the complexity in evaluating the outcomes from all possible conditions likewise increases. Figure 4 presents the number of possible conditions if there are three properties included in the model and two property variations: autonomous and non-autonomous. The graph shows that the number of conditions is exponentially grows as more stakeholders are involved in the decision-making process.

This paper used one case study of a community bio-digester installation project and the number of stakeholders involved in the simulation is based upon the empirical case. Even though each person in a decision-making process can be counted as a stakeholder, some individuals belong to similar groups, have the same interests and power or influence which means they can be identified as one group of stakeholders or can be represented by one person acting as the leader of the group. The stakeholders included in the simulation in this paper are based on the particular case study described.

In the decision-making process on the community biogas system in Bandung, Indonesia, the stakeholders involved can be classified based on their role, interest, and power. The list of stakeholders involved in the decision-making process of the community biogas system are:

(1)

The RW officials, consisting of the RW chief and the people within the RW management, such as the RW vice chief, RW secretary, etc.

(2)

The members of the PKK of RW 11 (women organisation), which consists of women living in RW 11. The PKK contributes in segregating and recycling waste into handcrafts.

(3)

The members of Karang Taruna of RW 11 (youth organisation), which consists of youths, usually between 15 and 30 years old, living in RW 11. The members of this organisation mainly contribute in the operation and maintenance of the composting/bio-digester facility.

(4)

The members of Forum RW 11, a community-based organisation specially formed for this project. This Forum actually consists of the RW officials and other concerned or interested community members. This Forum is a place for gathering ideas and making decisions, especially regarding sanitation matters.

(5)

The members of Cibangkong 11 Co-op, a cooperative formed for the composting project. This co-op handled the marketing of the compost product. However, since the composting centre was terminated, the co-op then became deactivated.

(6)

The Sub-district officials (Kelurahan), as part of the local government.

(7)

Bandung City Sanitation Agency, as part of the local government.

(8)

Housing Research and Development Centre (Puslitbangkim), Indonesian Ministry of Public Works, which represents the national government interest.

(9)

Experts from Padjadjaran University, which represent academicians.

(10)

Bank of West Java as an investor.

Among the ten stakeholders above, the first five stakeholders are comprised of the local people who live in RW 11. Whereas stakeholders 6–10 are people from outside the community, but they have interest or influence in this activity.

Based on the decision-making process investigated regarding the biogas project, there are at least four stakeholders who contributed highly and actively towards the success of this waste-to-energy activity over the period of 15 years (begins in 1999, until time of field work in 2014). The CBIIP is a program initiated by the Housing Research and Development Centre, Indonesian Ministry of Public Works, therefore they are one of the main stakeholders in this sanitation-related activity. The role of Bandung City Sanitation Agency is also quite prominent in terms of giving consultancy and marketing to the product. The Sub-district officer of Cibangkong is important as the supervisor of the activity and as the lowest formal institution in the society, they can assist with legitimisation through the required legal procedures. Both the Sub-district officials and Sanitation Agency, represent the interests and authority of the local government of Bandung City.

The other two main stakeholders are the community chief and the experts. The community chief plays an important role as the leader of the community, he has great influence on the community attitude and behaviour towards the project. He also becomes the contact person and intermediary between the local people and the “outsiders”. Other stakeholders within the RW 11 community are important, but in reality, their roles and contributions are influenced by the RW chief or leaders. Therefore for the model, the community behaviour is represented by only one stakeholder, which is the chief.

The experts or academicians from Padjadjaran University in this case perform at least two roles that contribute to this project. The first role is to provide technical consultation and conduct research as part of the project construction and operation. The second is providing consultation to the community, such as assistance in writing proposals for investors and so on, even after the project is terminated. In this case, the academicians are not representing any parties or stakeholders, therefore they can be seen as an independent stakeholder.

The last stakeholder is the Bank of West Java, which was involved as the investor of the project. The bank is only involved in the later part of the project, especially after the project fund from the Puslitbangkim of Ministry of Public Works is finished and terminated. The bank would not become involved if the community does not sent the proposal to them. Therefore, this stakeholder is not considered as a main stakeholder in this case.

Based on the stakeholders’ roles explained above, it is evident that the main stakeholders in this case can be grouped into four representative stakeholders: (1) National Government; (2) Local Government; (3) Community; and (4) Academicians. These four stakeholders are included in the model simulation for evaluating the performance of autonomous decision-making processes.

3.1.2. Decision-Making Conditions

This section describes how decision-making conditions are formed, the classification of the decision-making conditions into three categories based on the level of autonomy exhibited. Decision-making conditions are built through combining the stakeholders’ properties. As previously mentioned, there are seven stakeholders’ autonomous properties identified. For the simulation purpose, we selected only several important properties in decentralized decision-making. Malone (2004) stated that decision information, trust, and motivation are among the important factors necessary for applying decentralized decision-making [23]. On the basis of this source, the autonomous properties selected for the simulation are: (1) motivation or initiative; (2) leadership; and (3) ability to manage information. Moreover, the analysis of five various decision-making cases showed that a proactive community (leaders and members) contributes to the success of a project. The proactive trait is an actualization of two properties combined, which are motivation or initiative, and leadership. This is one of the reasons why these two properties are among the first to be included in the simulation.

The persuasion and negotiation ability property is not included in the simulation because we consider these abilities are at least partially accommodated under the leadership property. Meanwhile, the (internal) interaction, and external interaction (networking, collaboration) properties are not included in the simulation because the Petri-net itself is modelling the interaction between the stakeholders involved. Hence, these properties are the basic requirement for the model. The last property, responsibility and commitment, is also a basic requirement of the model, because there is no case that can successfully reach a decision and be implemented without the responsibility and commitment from its stakeholders (although various levels may be ascribable). Therefore, for the sake of demonstration the three selected properties can cover most of the autonomous behaviours in individuals or parties.

The three selected autonomous properties, are then embedded into each stakeholder involved in the decision-making process. The definition of each property differs from one stakeholder to another, which will then influence the outcome of the decision. Although the simulation in this paper only employed three properties, the developed Petri-net model is flexible and can be changed to accommodate other properties if deemed necessary.

As mentioned above, each stakeholder has three properties, namely: motivation, leadership, and ability to manage information. For each property, it is divided into two possible mutually exclusive conditions, simply the more autonomous condition (such as high motivation, strong leadership, and high ability to manage information); and non-autonomous condition (low motivation, weak leadership, and lack ability to manage information). This means, if a stakeholder is assigned as having a high motivation, then there is no possibility for that particular stakeholder to also have the property of “low motivation”, and that particular property (the motivation level of the stakeholder) will stay the same throughout the simulation.

Among various stakeholders that were involved in the case study, we selected the six most common stakeholders that are frequently involved in development projects [24]: national government, local (city) government, community leader, interested individual, academician/expert, and private sector. The definition for each stakeholder to be classified as autonomous or not-autonomous might be different. For example, the motivation for national government or local government to initiate a development project might be budget availability and national regulations. However, the community leader or interested individual, might not be motivated because of those things. To provide clarity on the definition of autonomous conditions, we classified the autonomous level based on the stakeholders’ properties as follows:

(1)

Autonomous condition occurs when at least two stakeholders have all three autonomous properties (motivation, leadership, and the ability to manage information).

(2)

Non-autonomous condition occurs when only one stakeholder has a maximum of all three autonomous properties.

(3)

Semi-autonomous condition is a situation when the stakeholders hold any of the properties other than the two defined above.

The detailed definitions and variations of stakeholders’ conditions are listed in

Table A1. Variations of stakeholders’ autonomous properties and detail definition.

Ability/Conditions	Definition	Property Variations
National Government
Motivation	Motivation for the national government to perform something/some projects is caused by the existence of a budget for the project, an existing plan already established, and political pressure/political will.	High Motivation: If the national government has budget allocation for the project, or political pressure from the public, or already has higher commitment/regulated plan. Low Motivation: If the national government does not have a budget allocation, or no political pressure from other stakeholders, or does not have any commitment/regulated plan.
Leadership	Leadership for the national government is the ability to coordinate their subordinates, and facilitate communication with other parties including the national government, private sectors, citizen, experts, and media.	Strong leadership: If the NG can enforce their power with their subordinates, or can communicate and coordinate with other stakeholders. Weak leadership: If the NG cannot enforce their power with their subordinates, or cannot communicate and coordinate with other stakeholders.
Ability to manage information	Capacity to collect and understand information without help from other parties.	High ability to manage information: Able to collect information, understand information, etc. Low ability to manage information: Does not know which information is valuable and which information need to be collected.
Local Government
Motivation	Motivation for the local government to perform something/some projects is caused by the existence of budget for the project, an existing plan from higher authorities, and political pressure/political will.	High Motivation: If the local government have budget allocation for the project, or political pressure from the public/NG, or already have higher commitment/regulated plan. Low Motivation: If the local government does not have budget allocation, or no political pressure from NG/other stakeholders, or does not have any commitment/regulated plan.
Leadership	Leadership for the local government is the ability to coordinate their subordinates, and facilitate communication with other parties including the national government, private sectors, citizen, experts, and media.	Strong leadership: If the LG can enforce their power with their subordinates, or can communicate and coordinate with NG and other stakeholders. Weak leadership: If the LG cannot enforce their power with their subordinates, or cannot communicate and coordinate with NG/other stakeholders.
Ability to manage information	Capacity to collect and understand information without help from other parties.	Capable to manage information: Able to collect information, understand information, etc. Low ability to manage information: Does not know which information is valuable and which information need to be collected.
Community Leader
Motivation	Motivation for the community leader to perform something/some projects is caused by the existence of a budget for the project, an existing plan from higher authorities, aspirations from the community members, and individual concern/motivation.	High Motivation: If the community (RW) leaders have budget allocation for the project, or pressure/existing plan from higher authorities, or aspirations from community members, or personal concern/experience/proximity. Low Motivation: If the community (RW) leader does not have budget allocation for the project, or no pressure/existing plan from higher authorities, or lack of aspirations from community members, or no personal concern/experience/proximity.
Leadership	Leadership for the community leader is having the ability to organize/coordinate the community members and facilitate relationship and collaboration with other parties.	Strong leadership: If the CL has the ability to organize the community members, or has the ability/willingness to communicate and collaborate with other stakeholders. Weak leadership: If the CL does not have the ability to organize the community members, or does not have the ability/willingness to communicate and collaborate with other stakeholders.
Ability to manage information	The situation of having knowledge and the ability to compile sufficient information about the situation/problem in their community.	High ability to manage information: Able to collect information, understand information, etc. Low ability to manage information: Does not know which information is valuable and which information needs to be collected.
Interested Individual
Motivation	Motivation for the interested individual is caused by personal concern, experience, and/or education, which leads them to perform some activities to solve the problem.	High Motivation: If the person has personal background/experience/education, or personal concern, or proximity to the problem/affected directly about the problem in the community. Low Motivation: If the person does not have personal background/experience/education, or has no personal concern, or is distant from the problem (is not affected directly from the problem).
Leadership	Leadership for an interested individual is having the ability to persuade the community members and posit themselves as a front liner/figure in the community and facilitate communication process to other people.	Strong leadership: If the person has the ability to become an example for the community/posit themselves as a front liner, or has the ability to persuade other members, including other stakeholders. Weak leadership: If the person hardly ever becomes an example for the community/posit themselves as a front liner, or does not have the ability to persuade other members, including other stakeholders.
Ability to manage information	The situation of having knowledge and ability to understand about the situation/problem nearby/in the community.	High ability to manage information: Able to collect information, understand information, etc. Low ability to manage information: Does not know which information is valuable and which information needs to be collected.
Private Sector
Motivation	Motivation for the private sector is the availability of budget (Corporate Social Responsibility), pressure from the government/public, economic benefit	High Motivation: If the firm/company have budget available for CSR projects, or pressure from public/government, or sees an economic benefit from the project. Low Motivation: If the company does not have a CSR budget available, or no pressure from the public/government, or sees no economic benefit to do the project.
Leadership	Leadership for the private sector means they have the ability to persuade other parties and facilitate collaboration and networking	Strong leadership: If the company has the ability to persuade other stakeholders to participate in their project. Weak leadership: If the company does not have the ability to persuade other stakeholders to participate in their project.
Ability to manage information	Capacity to collect and understand information without help from other parties.	High ability to manage information: Able to collect information, understand information, etc. Low ability to manage information: Does not know which information is valuable and which information needs to be collected.
Expert/Academician
Motivation	Motivation for the experts/academician is education background, responsibility to the community/government, personal experience, and proximity to the community/to the problem	High Motivation: If the experts/academician has a suitable education background, or has personal concern/responsibility to the community/government, or has proximity to the community/problem. Low Motivation: If the experts/academician does not have a suitable education background, or does not have personal concern/responsibility to the community/government, or is distant from the community/problem.
Leadership	Leadership for the experts/academician means they have the ability to persuade other parties and facilitate the collaboration and networking, based on the knowledge and expertise they have.	Strong leadership: If the experts/academician have the ability to persuade and convince other stakeholders, or have the ability to become example for other stakeholders. Weak leadership: If the experts/academician have the ability to persuade and convince other stakeholders, or does not have the ability to become an example for other stakeholders.
Ability to manage information	Capacity to collect and understand information and analyse based on their expertise.	High ability to manage information: Able to collect information, understand information, etc. Low ability to manage information: Does not know which information is valuable and which information needs to be collected.

.

3.1.3. Performance Indicators

Performance indicators are utilised to assess the decision-making results. Based on the critical points referred in the end of Section 2.2, we defined six stages of decision-making outcomes that are possible to be reached by the net. The stages are:

(1)

Problem not defined, occurs when T4 fails to fire. At this stage, no decision is made, therefore the decision outcome fails.

(2)

No solutions designed or proposed, occurs when T5 fails to fire. At this stage, no decision is made, therefore the decision outcome fails.

(3)

Consensus is not reached, occurs when T6 fails to fire. At this stage, no decision is made, therefore the decision outcome fails.

(4)

Project failed to be managed, occurs when T8 fails to fire. At this stage, a decision is made but the implementation fails, therefore the decision outcome fails.

(5)

Project is moderately successful, occurs when the net ends at P10. At this stage, the net reaches the end, the decision outcome is a moderate success.

(6)

Project is fully successful, occurs when the net reaches P11. At this stage, the net reaches the end, the decision outcome is successful.

Based on the six stages above, the decision outcomes in the autonomous decision-making model are considered as:

(1)

Failed, meaning that there is no decision produced, or there may even be no problem recognised or no solutions proposed (Stages 1–3).

(2)

Moderate success, meaning the project is constructed physically, however the community is not involved sufficiently and therefore it is difficult to continue operating the project, both during the project term and after the budget is finished (Stages 4 and 5).

(3)

Success, meaning the project is executed or constructed physically. In the decision-making and implementation phase, the community is actively involved and able to operate or regenerate beyond the project term (Stage 6).

The simulation results are categorised based on these indicators and the autonomous level, and then the conditions underlying each category are identified.

3.2. Simulation of the Autonomous Decision-Making Model

An objective evaluation of the autonomous decision-making performance is performed by simulating the Petri-net model presented in Figure 3. The basic principle of the simulation is to change the conditions of the decision-making and record the decision-making result under those particular conditions. The general autonomous decision-making model is designed hierarchically in order to be able to easily modify each of the stakeholders’ properties. As explained in Section 3.1.2, there are two variations of possible properties: autonomous and non-autonomous. In the model, the properties are assigned to each stakeholder at the beginning of the simulation (T1–T3). Each condition for each stakeholder is mutually exclusive (it cannot hold multiple states). The number of stakeholders involved and the property of each stakeholder are defined from the beginning of the simulation, and cannot be changed during the simulation. For example, if the stakeholder “National Government” has a low capability to manage information, then it cannot be changed or modified throughout the course of the simulation. The simulation follows the concept presented in Figure 5.

If all stakeholders’ properties are assigned and combined, it is regarded as one condition that can be simulated. The result of the decision-making process under that particular condition is then recorded. This process is then repeated many times with different combinations of stakeholders’ properties. There are thousands of possible variations of conditions, and this increases exponentially if the number of stakeholders is increased (see Figure 4). For this reason, a stochastic simulation was selected. The combinations of stakeholders’ properties are randomised, with the probability of each stakeholder to have or not have the autonomous properties assumed to be equal. This might not be perfectly representative of the real situation (i.e., some stakeholders will have inherent tendencies towards autonomy that make the probability different from 50%), however it is considered sufficient to demonstrate the evaluation method.

In order to demonstrate the method, the expanded model is simulated stochastically following the concept in Figure 5. Based on the previous explanation of the elements of evaluation, four stakeholders are employed in the simulation. In this paper, the number of simulations performed is 50. The results of each simulation are recorded and afterwards the distribution of results are separated into various levels of success and reported as a percentage of the total trials. In order to obtain a stable percentage and provide a sensitivity analysis, 20 sets of 50 simulations were performed. After performing 20 sets of simulations, the percentage of each stage is relatively stable, therefore we considered that the number of simulations was sufficient to provide robust analysis. The average of each 5 sets of these simulations is presented in

Table 3. Simulation Result.

No.	Stage	Average Percentage/Simulation				Average Cumulative Per Outcomes
		5	10	15	20
1	Problem not defined (T4 failed)	13.6%	12.6%	12.7%	12.6%	Failed: 37.6%–41.1%
2	Alternatives not designed (T5 failed)	20.0%	24.2%	24.0%	24.8%
3	Consensus not reached (T6 failed)	4.0%	3.2%	3.5%	3.7%
4	Project finished but failed to be managed (T8 failed)	10.0%	10.2%	10.1%	10.6%	Moderate success: 23.8%–26.0%
5	Project moderately success (simulation ended at P10)	16.0%	14.0%	13.9%	13.2%
6	Project successfully implemented (simulation ended at P11)	36.4%	35.6%	35.7%	35.0%	Success: 35.0%–36.4%

, showing the level of variability.

For the simulation, we created a simple program in the Python language. There are two benefits of creating a specific program for modelling and simulating Petri-net. Firstly, we can determine the different rules for each transition in Petri-net. Secondly, considering that our Petri-net model is a simple one, we can tailor the program to fit our purpose without using the complicated features that often come with existing popular programs for Petri-net.

4. Results and Discussion

The purpose of the model is to evaluate autonomous decision-making performance. The model is designed to be adjustable; therefore, there is actually no limitation on the number of stakeholders involved, the number of properties, and set of conditions included in the simulation. Each combination can lead to producing a decision, or to the process failing to reach a decision, meaning the Petri-net simulation does not reach the end of the network. The more stakeholders involved means more sets of stakeholders’ properties combinations are possible. Therefore, the stochastic method is selected to simulate the model.

As explained earlier in the paper, there are four stakeholders selected to be included in the model simulation: (1) National Government; (2) Local Government; (3) Community; and (4) Academicians. These four stakeholders are the main stakeholders that have significant contributions towards the outcome of the project. In addition, these four stakeholders are among the six key decision-makers that are frequently involved in development project identified by Sexton et al. [24] (p. 4), which are: national governments; regional or local government bodies; business associations; environmental advocacy groups; community or neighborhood groups; and affected or interested individuals. In other case studies, the number and type of stakeholders involved might be different to those identified from the biogas case.

The stochastic simulation is utilised despite the number of possible combinations of four stakeholders being 4,096 conditions, at which level it is still possible to undertake the simulation for each condition. In this way, the demonstrated method can be applied directly with the addition of more conditions or stakeholders. In order to test the stability of the model, the simulations were performed 20 times each over 50 sets of randomly-assigned conditions. The average of the 20 simulations are presented in Table 3.

Table 3 shows the percentage of simulations that demonstrated failure (Stage 1–3), moderate success (Stage 4 and 5), and success (Stage 6) for every five sets of simulations. It is shown that the percentage of respective simulations is relatively stable. The last column of Table 3 indicates that around 40% of the simulations failed to reach a decision (Stage 1 to Stage 3), whereas the rest (60%) reached a decision representing either moderate success or success. If the simulation ended between Stage 1 and 3, it means the particular conditions could not achieve a consensus or produce a decision. In addition, there is a large difference between the number of conditions that reached Stage 2 and Stage 3. This could mean that, if there is an alternative solution was designed or proposed, then the possibility of reaching a decision by consensus could be very high. Lacking the “ability to manage information” or “self-learning” property is the reason for this outcome. In order for a decision to be reached, there is a need for information sharing and collection and learning from that shared information. If the stakeholders lack this ability, then the sharing and learning process do not occur, which eventually leads to no solution being proposed.

Meanwhile if the simulation ended at Stage 4 (±10%), the decision is reached and implemented, but fails to be operated or managed after some time due to a variety of reasons. Alternatively, there are two possible stages reaching the end of the simulation, which are Stage 5 (±14%) or Stage 6 (±35%). Stage 5 is still considered as moderate success because the condition is able to reach a decision, be implemented, and operated, however after the project budget is terminated, the project could not survive and would be discontinued. Stage 6 means the project is successful and sustained even after the project term is finished.

As explained in the Section 3, the simulation results are further classified based on the autonomous level and the decision-making outcomes. There are various levels of autonomous properties for each of the stakeholders that lead to the respective outcomes, and they can be classified into six categories as follows:

(1)

A successful decision outcome under autonomous conditions.

(2)

A successful decision outcome under semi-autonomous conditions.

(3)

A moderately successful decision outcome under autonomous conditions.

(4)

A moderately successful decision outcome under semi-autonomous conditions.

(5)

A failed decision outcome under semi-autonomous conditions.

(6)

A failed decision outcome under non-autonomous conditions.

Theoretically, there are other possible categories: (1) a successful decision outcome under non-autonomous conditions; (2) a moderately successful decision outcome under non-autonomous conditions; and (3) a failed decision outcome under autonomous conditions. However, the rules imposed to the model prevented these outcomes, hence there are only six possible categories identified here.

In addition, the simulation results are classified based on the conditions and the decision-making outcomes are analysed. The first step performed is calculating the number of autonomous conditions, semi-autonomous conditions, and non-autonomous conditions simulated in one series. This procedure resulted in the identification of 320 simulations of autonomous conditions, 515 for semi-autonomous conditions, and 165 non-autonomous conditions as presented in

Table 4. Simulation Results Based on Level of Autonomy.

No.	Conditions	#Simulation	Percentage
1	Autonomous condition	295	29.5%
2	Semi-autonomous condition	518	51.8%
3	Non-autonomous condition	187	18.7%
	Total	1000	100%

.

The second analysis further examined each condition and investigated the outcomes of each category. Since the non-autonomous conditions can only reach one outcome, which is failure, therefore this category is not presented here. The other two conditions can meet various outcomes, and they are presented in the following tables.

From the tables above, it is shown that the probability of autonomous conditions reaching successful outcomes is very high (

Table 5. Outcomes under Autonomous Conditions.

DM Outcomes	#Simulation	Percentage	Ratio
Successful	228	77.3%	3.4
Moderate success	67	22.7%	1
Total	295	100%

), whereas the probability of semi-autonomous conditions reaching successful outcomes or moderately successful outcomes is relatively lower compared with those that reach failure outcomes under similar conditions (

Table 6. Outcomes under Semi-Autonomous Conditions.

DM Outcomes	#Simulation	Percentage	Ratio
Successful	126	24.3%	0.7
Moderate success	168	32.4%	1
Failed	224	43.2%
Total	518	100%

). It is apparent from the ratios of Successful to Moderately Successful outcomes presented in the tables that it was more likely for Autonomous conditions to produce a success than Semi-Autonomous.

A more detailed examination of the properties combinations of each stakeholder and the corresponding outcomes is classified in

Table 7. Detail Conditions of Decision-making Outcomes.

Autonomous Condition	Semi-Autonomous Condition	Non-Autonomous Condition
Success outcomes would be reached when: Both the NG and LG have all three properties, and either C or A at least has leadership. Both C and A have all three properties.	Success outcomes would be reached when: Either NG or LG have all three properties combined, and either C or A also has all three properties. Either NG or LG have all three properties combined, and both C and A at least have motivation and leadership combined. Either C or A have all three properties combined, and both NG and LG has motivation; or leadership; or ability to manage information. Both NG and LG have motivation and either C or A at least has leadership. Both NG and LG have leadership and both C and A at least have motivation or leadership combined. Both NG and LG have ability to manage information and both C and A at least have leadership and motivation combined.	-
Moderate success outcomes would be reached when: Both the NG and LG have all three properties, and either C or A only has (at most) motivation or ability to manage information. Both the NG and LG have all three properties, and both C or A do not have any property at all.	Moderate success outcomes would be reached when: Either NG or LG have all three properties combined, and either the C or A only has (at most) motivation or the ability to manage information. Both NG and LG have motivation, and either the C or A only has (at most) motivation or the ability to manage information. Both NG and LG have leadership and either the C or A only has (at most) motivation; or leadership; or the ability to manage information; Either C or A have all three properties combined and both the NG and LG only have (at most) ability to manage information.	-
-	Failed outcomes would be reached when: Only one stakeholder has the ability to manage information.	Failed outcomes would be reached when: Only one stakeholder has (at most) all three properties.

Abbreviation: NG = National Government; LG = Local Government; C = Community Chief; A = Academician.

. The information in Table 7 can be used to understand the necessary properties of the stakeholders in order to create a successful outcome or avoid a failure. For example, in order to improve the outcomes of semi-autonomous conditions towards successful outcomes, fundamentally the “leadership” property is required, especially in the community. In a community project, the community is the main actor, therefore their active participation is needed. However, in the Indonesian context, the community members usually look up to a trusted person who can become their leader, in either a formal or informal setting. This is the main factor to increase the success rate of an innovative project.

Another example, by looking back to the case study, was the importance of the “ability to manage information” or “self-learning” property, which can be further discussed. After the composting centre experienced stagnancy due to lack of funds or profit, the RW 11 community shared their experience and project reports with the academicians. This further led to many discussions about lessons learned from the composting centre and eventually another technique was proposed. Therefore, in order to avoid the probability of reaching unsuccessful outcomes under the semi-autonomous conditions, there are at least two key stakeholders that need to have the ability to manage information and share or learn from the information.

At the level of the government entities, a budget or motivation is also a crucial factor—without which, it is unlikely the project will receive sufficient attention. This result argues fundamentally for a bottom-up, locally-focussed approach to ensure that the leadership, motivation and information are available in the community and with the local level government, which combined can produce many of the successful outcomes.

5. Conclusions

This study has developed a method for evaluating autonomous decision-making performance and applied it to a case study to further examine the influence of various possible conditions and their results. The evaluation method developed in this study has two main requirements, the decision conditions and the performance indicators. The decision conditions are derived from a combination of stakeholders’ properties extracted by decomposing the decision-making process, and are mainly employed in the Petri-net model simulation. The performance indicators are defined from critical points in the decision-making process, and are utilised for evaluating the simulation results. The whole evaluation process covers five steps:

(1)

Decomposing a decision-making process into decision-making events/stages.

(2)

Developing an expanded decision-making model using discrete event system tool.

(3)

Extracting stakeholders’ autonomous properties.

(4)

Simulating the model under various stakeholders’ conditions (combinations of properties) stochastically.

(5)

Assessing the simulation results based on performance indicators (success, moderate success, failure).

The simulation results showed that a decision-making process performs better if the key stakeholders exhibit autonomous behaviour. The results have also shown that a semi-autonomous condition is good enough to reach a successful decision-making. However, it requires a specific combination of stakeholders’ properties from particular stakeholders.

Through the simulation, the performance of the method is evaluated, and the method to evaluate autonomous decision-making performance is also demonstrated. The analysis of the simulation results also demonstrates how to identify conditions for certain decision outcomes. The simulation results would be different when the number, type, and characteristic of stakeholders involved are modified. Similarly, the results would also be different if the properties included are changed. These changes would affect the rules for the Petri-net transitions. However, the principals and basic methods to apply the stakeholders’ conditions and transitions rules would be generally applicable as demonstrated in this paper. The method to assess the decision outcomes was also performed through classifying the results into several categories based on the properties and decision-making outcomes.

This study has proposed an evaluation method for a new concept of autonomous decision-making in energy and environmental innovations. This paper has demonstrated an example to expand a decision-making model based on a case study, and has shown how to identify the underlying conditions of certain decision outcomes. The expanded model developed here has the ability to accommodate more properties and stakeholders. The evaluation method shown in this paper has also yielded some conditions in which autonomous decision-making performance is suitable for reaching the goal in community energy and environmental innovation projects.

The model developed and simulated in this study has some limitations. The current model does not include the variable of “time”, and the assumption taken in the simulation is that the stakeholders’ properties do not change throughout the decision-making process. By factoring in the time variable, the stakeholders’ change of mind, change of behaviour, and their learning process can be represented in the model and thus, will create changes in other stakeholders’ properties in the simulation. If the time variation is factored in, these stakeholders’ properties might change, hence mirroring more closely with the real world situation, where people could change their behaviours, motivation, skill, and ability throughout the time period due to learning and education. This would specifically represent the property of “ability to manage information” or “self-learning” better.

The purpose of this model is specifically to evaluate decision-making processes which exhibit autonomous behaviour. This model can also be utilised to estimate the likely successful outcome of an innovative activity in the community or a certain area, coinciding with a certain improvement project. In order to do achieve that purpose, the visionary model would be a model that can estimate the outcome of a project or decision by taking quantitative input of complex and varied stakeholders’ properties and contextual information. Having such a model would make the decision-maker or policy-maker better able to predict the outcome of a project under various stakeholders’ capacities, and understand which stakeholder and what capacity should be improved in order to reach a successful goal. The information and data on the stakeholders’ capacities, of course, require a deep and thorough analysis on the behaviours, capacity, ability, and their characteristics.