Development of a Sustainability Index for an Energy Management System in Thailand

Abstract

The ISO 50001 energy management system (EnMS) standard was published in June 2011 and has been widely adopted by organizations from around the world, including Thailand. From 2014–2017, there was a continuous increase in the number of ISO 50001-certified companies in the East Asia and Pacific regions and, more broadly, the world, although this is not consistent with the number of companies that emerged during this period in Thailand. This information shows that the implementation of energy management in some companies may not be sustainable. This research offers a novel method for assessing the quality of energy management in the form of an energy management system sustainability index (EnMS SI) framework, presenting the economic, organizational, energy performance, and environmental aspects of sustainable energy management. Data collection, from a literature review of related research and the EnMS good practices, was implemented in order to select sustainability indicators and further develop a sustainability index for energy management. The analytic hierarchy process (AHP) and weighted arithmetic mean (WAM) were used to establish an EnMS SI. The study results were then assessed and validated using 31 ISO 50001-certified companies in Thailand. Direct interviews and questionnaires were used to obtain responses from energy management representatives. The studied data indicated that an EnMS SI framework can be used in qualitative analyses to effectively determine the sustainability of an EnMS. Significant sustainability indicators, consisting of continuous benefits, top management commitment, and long-term strategic planning, were found. The results also revealed that the EnMS in Thailand has been significantly economically weak. The EnMS SI framework is a tool for assessing energy management sustainability, which allows for the determination of an organization’s actual strengths and weaknesses. The benefits of this framework include the possibility of determining guidelines for correcting and improving the EnMS to achieve sustainability.

1. Introduction

An energy management system (EnMS) is an important tool adopted [1] in the industrial sector to improve efficiency [2] and to reduce costs such as in cement industy [3], the energy intensive industries in Turkey [4] and Taiwan [5], and in Serbian manufacturing [6], as well as in mitigating the environmental impact of climate change [7]. The use of fossil fuels for energy is still the major cause of environmental problems and climate change [8], especially in industrial production. The implementation of energy-efficient projects has had a direct impact on reducing costs in various industries, i.e., the cement industry in Colombia [3], China [9], and Taiwan [10], and pulp and paper [11], food [12], sugar [13], and foundry industries [14]. For example, operational guidelines associated with energy management, such as ANSI/MSE 2000:2008 [15], EN 16001 [16], and Carbon Trust (2011) [17], were employed to mobilize and improve energy efficiency. On 15 June 2011, the International Organization for Standardization (ISO) published ISO 50001 for energy management [18], which has since been widely adopted around the world [19]. With regard to energy management in Thailand, the Energy Conservation Promotion Act (No. 2) B.E.2550 (2007) [20] was issued to encourage energy efficiency in designated buildings [21] and designated factories [22] by the energy management procedure [23]. Information from a survey on the number of ISO 50001-certified organizations [19] is shown in Table 1. ISO 50001-certified organizations in Thailand were found to have increased from 10 in 2011 to 168 in 2014, although this figure subsequently decreased and increased alternately, which is inconsistent with the situation in the East Asia and Pacific regions and, more broadly, the world, which all show a continuous increase. This information indicates that, although a number of organizations are ISO 50001 certified each year, they do not maintain an effective energy management protocol or implement it sustainably. However, organizations with ISO 50001 certification are predicted to continually increase in the future [24].

Table 1. The number of organizations in Thailand with ISO 50001 certification from 2011–2017.

Year	2011	2012	2013	2014	2015	2016	2017
The World	459	2236	4826	6765	11,985	20,216	21,501
East Asia and the Pacific	49	191	478	693	1035	2086	2516
Thailand	10	41	132	168	138	255	216

There have been numerous studies on the significant barriers to energy efficiency in many countries. For example, a study on the main barriers to energy efficiency in Singapore-based industries revealed that a firm’s energy efficiency outcomes are not significantly related to its corporate social responsibility, regulatory compliance, and capacity to implement energy efficiency [25]. A study on the foundry industries of Finland, France, Germany, Italy, Poland, Spain, and Sweden revealed that almost half of these countries lacked a long-term energy strategy [26]. In Europe, numerous studies on the potential to implement long-term energy strategies revealed that the barriers to energy efficiency include a lack of information, procedural impediments, and lack of energy efficiency awareness [27]. In the Swedish iron and steel industry, the most important barriers relate to internal economics and behavior [28]. In Thailand, the most significant barrier to energy efficiency appears to be that management is more concerned with production and other matters, rather than energy efficiency [29]. However, it is quite difficult to ascertain which significant barrier actually causes some organizations in Thailand to stop adopting ISO 50001.

The first edition of the ISO 50001 standard [18] was published in 2011. Subsequently, in 2014, guidance for the implementation, maintenance, and improvement of an EnMS and the measurement and verification of energy performance (ISO 50004 [30], ISO 50006 [31] and ISO 50015 [32]) were introduced so that organizations could apply them effectively. The principle underlying these standards is the Plan–Do–Check–Act (PDCA) cycle. Many studies have proposed various guidelines to facilitate and develop an energy assessment methodology for ISO 50001 such as Chiu et al. (2012) [33], Gopalakrishnan et al. [34], and Kanneganti et al. [35]. In addition, an energy management maturity model [36] can be used to guide organizations in their energy management implementation efforts in order to incrementally achieve compliance with standards. The maturity model concept, introduced by Shewhart [37] and Crosby [38], uses grids for the economic control of quality [38]. A considerable number of studies have offered various guidelines for implementation, while the principles of the maturity model have been applied to assess the success of systems and processes, such as energy efficiency [39] and manufacturing [40]. An EnMS was applied to support implementation and assess success from the beginning of the process until certification (EMMM). O’Sullivan (2010) assumed that successful energy management could cause considerable savings in terms of energy, among other things, and provide future benefits, since the system would be rooted deeply in the process of business and operational management in an appropriate manner [41]. Later, Antunes et al. (2014) proposed an organizational guideline to allow energy management to be practiced, and measured the success by implementing the ISO 50001 standard [36]. Jovanovic and Fillipovic (2016) proposed a maturity model based on the PDCA cycle guideline, and integrated the requirements of ISO 50001 for energy management with the capability maturity model (CMMI), with the objective of using it as a tool to help organizations achieve a higher energy performance than the standard level (EMMM 50001) [42].

ISO 50001 is a process standard, not a performance standard [7]. Therefore, the assessment of certified energy management standards is considered the success of the process, according to the energy management requirements [43,44]. In the meantime, a study by Pham (2015) revealed that the adoption of ISO 50001 has quite a negative effect on the market. This result does not suggest that obtaining ISO 50001 is a bad investment [45]. To date, no guidelines can be used to measure the qualitative results of an EnMS and, thus, indicate its effectiveness and anticipated benefits in order to show the quality or sustainability levels of an organization’s EnMS. However, the second edition of ISO 50001 was published in August 2018 [46], based on the existing PDCA cycle guideline, but with a new clause arrangement, adding an analysis of the organizational context for consistency with ISO 9001 [47] and ISO 14001 [48]. This allows for a simple integration of management systems, in line with the organizational business strategy. However, this study does not include the implementation or adoption of the new version of the standard by organizations.

The concept of an EnMS sustainability index (SI) refers to “Our Common Future” published by the WECD, which offers a definition of sustainable development as that which “meets the needs of the present without compromising the ability of future generations to meet their own needs” [49]. This fundamental requirement was the starting point for developing concepts for the implementation of sustainability initiatives [50]. Sustainability is a paradigm, with different implementations and interpretations in various fields [51], such as the energy system [52], automotive industry [50], waste to energy system [53], renewable energy projects for energy and sanitation devices [54], and local communities [55].

In this context, the sustainability index is introduced as an agglomerated indicator for the measurement of quality in an EnMS [56]. This proposed criterion for EnMS sustainability reflects the economic, organizational, energy performance, and environmental aspects of sustainable energy management. Indicators are considered within activities, under the control of an organization [18,46], involving the relationship between the effectiveness of an EnMS, business strategy integration, and environmental connectivity [7]. The significant barriers to and drivers of the improvement of energy efficiency, such as a gap analysis between industrial needs and scientific literature [57], the driving forces for improved energy efficiency in the foundry industries [26], the barriers and drivers to energy management practice in the Swedish iron and steel [28] and foundry industries [58], indicate the sustainability of the qualitative criteria in the assessment of an EnMS, according to the following aspects, shown in Table 2.

Table 2. Energy management system (EnMS) conceptual sustainability aspects.

Aspect	Economic	Organizational	Energy Performance	Environmental
Criteria	Investment and returns	Commitment	Energy planning	Environment impact
		Organizing ability	Implementation and operation
		Culture and behaviors	Performance evaluation

• Economic Aspect

The internal economics [28] associated with energy efficiency, such as investment and returns [7], can be measured by their impact on competitive advantage, triggered by the related costs [25]—for example, the benefit uncertainty [1], the financial support for energy management activities [4], and the limited to access to capital [58].

• Organizational Aspect

An EnMs involves a top-down approach. Therefore, the commitment of top management [18,46], energy policy [59], and long-term strategic planning [58,59] are key indicators of its effectiveness and sustainability, and consideration should also be devoted to power delegation [4] and resource provision [18,46]. This will help to support an EnMS by reflecting the awareness culture [27], behavior [28], and participation of personnel [25,58] in the implementation of organizational energy efficiency for long-term sustainability.

• Energy Performance Aspect

The continuous improvement of energy performance is a required outcome of the ISO 50001 standard [18,46]. Thus, the qualitative efficiency of the process, including energy planning [18,31,46], implementation and operation [18,30,46], and performance evaluation [32,46], has a significant impact on organizational energy performance.

• Environmental Aspect

Tangible outcomes of the integration of the environmental [48,60] and energy management processes [61] are considered in terms of waste management, resource reduction for low carbon [62], and climate change mitigation [24]. The activities that come under the control of an organization, excluding product design [46], such as eco-design [63], are considered in this context.

This paper proposes the development of an EnMS SI framework for assessing the quality and sustainability of energy management. Indicators that have a direct impact on energy performance are applied, using the AHP method and then evaluating the EnMS SI of each organization. This enables the evaluator to predict the potential sustainability of a current organizational EnMS and analyze its strengths and weaknesses. Important information for strategic decision-making to improve these systems and achieve sustainability can then be obtained.

2. Methods

The establishment of an EnMS SI can be divided into two major parts: The development of an EnMS SI framework and the application of that framework to assess ISO 50001-certified industries in Thailand for more than one year. The main objective of an EnMS SI is to provide comprehensive and highly scalable information-driven architecture for assessment [64]. The process, used to develop an EnMS SI framework, is shown in Figure 1 [50,55,64].

Figure 1. Development strategies for an EnMS sustainability index (SI).

2.1. Selection and Quantification of Basic Sustainability Indicators

The basic indicators of EnMS sustainability are the assessment tools for identifying its performance and trends, providing early warning information that supports decision-making [50]. The selection of sustainability indicators is based on the data collection of process factors and associated outcomes, which directly relate to organizational energy efficiency and EnMS effectiveness. The results of energy performance data analysis (Carbon Trust 2011 [17], ISO 50001 [18], ISO 50004 [30], ISO 50006 [31], ISO 50015 [32]) include those obtained from a literature review of related research on maturity model assessment [36,41,42] and study results concerning the impact of the main barriers and drivers on energy efficiency and energy management [1,4,25,26,27,28,29,57,58,59]. Both qualitative and quantitative indicators are available, as well as cover indicators, which can measure the qualitative results of the energy management process in organizations, starting with the input process indicator, work-in-process indicators, output process indicators, and outcome indicators.

Since the selected basic sustainability indicatiors (BSIs) have different dimensions, it is necessary to quantify them as non-dimension variables using the indicator value scale, 1, 2, 3 [64,65]. The qualitative value is assessed to obtain the quantitative value in the scoring of each indicator, and when the indicator values are in the same unit, normalization is not required. Here, the scoring criteria is determined using scales:

Scale	Meaning
3	Strong/excellent/maximum
2	Good/satisfactory/medium
1	Weak/unsatisfactory/minimum

2.2. Determination of the Indicator and Aspect Weights

In this paper, the analytic hierarchy process (AHP) methodology is used to obtain the indicator weights, and a case study, performed to illustrate the application of the suggested EnMS SI [50,64,66], is presented. The AHP method, developed by Saaty [67], involves the following steps.

(1)

Individual comparison matrices are collected from a questionnaire survey. The questionnaire survey is designed to produce individual comparison matrices by relevant experts in the field, as shown in Table 3.

Table 3. Pairwise comparison of the analytic hierarchy process (AHP).

Criteria	A	B	C	D
A	1	X12	X13	X14
B	1/X12	1	X23	X24
C	1/X13	1/X23	1	X34
D	1/X14	1/X24	1/X34	1

A, B, C, and D are the indicators, and X₁₂, X_13, and X₁₄ are compared to the value of the pairs AB, AC, and AD, respectively. The value is obtained from a basic range of absolute numbers in order to capture human perceptions regarding qualitative and quantitative attributes. The questionnaire is designed using the nine-point method, recommended by Saaty [67], and conforms to the paired comparison requirement of the AHP method, based on the final indicator list [64,66]. The values refer to the nine-point method, recommended by Saaty [67].

(2)

After constructing the preference matrix, the weights can be calculated using the maximum eigenvalue vector method (MEVM) [64,68]. In order to validate the reliability of the AHP model, a consistency ratio is used to measure the comparison matrix. To calculate the consistency index (CI), the deviation from consistency is given as [64,68]:

$$CI=\frac{\lambda max-n}{n-1}$$

(1)

where λmax is the maximum eigenvalue of the comparison matrix, n. There are n rows and n columns in a comparison matrix.

(3)

The random index (RI) is identified for consistency. The RI is provided by Saaty [67].

(4)

The consistency ratio (CR) is calculated.

The CR is calculated by Equation (2). Saaty stated that a CR of less than 0.1 is acceptable [67].

$$\mathrm{CR}=\frac{\mathrm{CI}}{\mathrm{RI}}$$

(2)

2.3. Sustainability Index Construction

To establish the sustainability index in order to obtain a final numerical result [64], the weighted arithmetic mean (WAM) method is used to aggregate the indicators into the indices. A linear agglomeration function is given in Equation (3).

Q (q; w) = ^m∑ w_iq_i
i = 1

(3)

where q_i is the value of the i-th indicator, and w_i is the weight of the i-th indicator.

A hierarchical scheme for the EnMS SI is shown in Figure 2. Figure 2 shows the economic (SI₁), organizational (SI₂), energy performance (SI₃), and environmental (SI₄) aspects, represented by the economic general indicators (GI_1,1 … GI_1,m) and basic indicators (BI¹_1,1…. BI¹_m,n), organization general indicators (GI_2,1 … GI_2,m) and basic indicators (BI²_1,1…. BI²_m,n), energy performance general indicators (GI_3,1 … GI_3,m) and basic indicators (BI³_1,1…. BI³_m,n), and environmental general indicators (GI_3,1 … GI_3,n) and basic indicators (BI⁴_1,1…. BI⁴_m,n) [50], respectively.

Figure 2. Hierarchical scheme for the EnMS SI.

For each company (A), the values of the EnMS sustainability index (SI_EnMS) is calculated by aggregating the corresponding basic indicators, general indicators, and each considered aspect, according to the following equations: Equations (5)–(7):

$$\left({\mathrm{GI}}_{\mathrm{i},\mathrm{m}}\right){}_{\mathrm{A}}=f[{{\mathrm{WB}}^{\mathrm{i}}}_{1,1}\times ({{\mathrm{BI}}^{\mathrm{i}}}_{1,1}){}_{\mathrm{A}},\dots ,{{\mathrm{WB}}^{\mathrm{i}}}_{1,\mathrm{n}}\times ({{\mathrm{BI}}^{\mathrm{i}}}_{\mathrm{m},\mathrm{n}}\left){}_{\mathrm{A}}\right]$$

(4)

where:

• (GI_i,m)_A is the value of the general indicator m, according to the EnMS sustainability aspect i for company A;
• (BIⁱ_m,n)_A is the value of the basic indicator n, corresponding to the general indicator m and associated with the EnMS sustainability aspect i for company A; and
• WBⁱ_m,n is the weight of the basic indicator n for the general indicator m, associated with the EnMS sustainability aspect i.

$$\left({\mathrm{SI}}_{\mathrm{i}}\right){}_{\mathrm{A}}=f[{\mathrm{WG}}_{\mathrm{i},1}\times ({\mathrm{GI}}_{1,1}){}_{\mathrm{A}},\dots ,{\mathrm{WG}}_{\mathrm{i},\mathrm{m}}\times ({\mathrm{GI}}_{\mathrm{i},\mathrm{m}}\left){}_{\mathrm{A}}\right]$$

(5)

where:

• (SI_i)A is the value of the EnMS sustainability aspect i for company A;
• (GI_i,m)A is the value of the general indicator m, corresponding to the EnMS sustainability aspect i for company A; and
• WG_i,m is the weight of the basic indicator m, corresponding to the EnMS sustainability aspect i.

(SI_EnMS)_A = ⁴∑ W_i(SI_i)_A
i = 1

(6)

where: (SI_EnMS)_A is the value of the EnMS sustainability index for company A, assuming values between 1 (less sustainable) to 3 (extremely sustainable). (SI_i)_A is the value of the quality of each EnMS sustainability aspect for company A, and W_i is the weight of each considered aspect.

2.4. Industry Sampling for Framework Validation

In order to verify the developed EnMS SI framework, questionnaires were used to assess the sustainability of indicators in the sampled companies (at least one year of certification with ISO 50001). The values obtained from the assessment were then calculated to achieve the EnMS SI of each company, with the outcomes analyzed and discussed accordingly.

3. Results

The implementation of the study outcomes and development of a framework in accordance with the process determined in Section 2 are shown below.

3.1. Selection and Quantification of the Basic Sustainability Indicators

The study and collection of the basic sustainability indicators follows the procedures mentioned earlier. Incidentally, a review of the process by eight specialists in the ISO 50001 field from public agencies, education institutions, professional associations, and the industrial sector, with direct experience of the ISO 50001 standard, was made possible by holding a focus group meeting to select 30 indicators for assessing EnMS SI, as detailed in Table 4.

Table 4. Basic EnMS sustainability indicators.

Aspect	General Indicator	Basic Indicator	Definition
Economic	Investment and returns [4,25,28,29]	BI11,1 Continuous benefit	Continuous benefit of an energy efficiency project.
		BI11,2 Energy cost proportion	Proportion of energy costs, reflecting the competitive advantage of an organization.
		BI11,3 Energy efficiency project investment	Proportion of investment in an energy efficiency project, compared to the total investment of an organization.
		BI11,4 EE investment procedure	Procedure and criteria for decision-making relating to investing in an energy efficiency project.
Organizational	Commitment [18,25,26,29,30,46,58,59]	BI21,1 Top management commitment	Tangible top management commitment to implementing an EnMS.
		BI21,2 Long-term strategic planning	Long-term strategic planning for efficient energy management.
		BI21,3 Energy policy	Policies and targets for organizational energy management.
	Organizing ability [4,18,27,30,46,57]	BI22,1 EnMS properly embedded into the business process	Relation and connectivity of energy management systems and the business operation process of organizations.
		BI22,2 Energy management and power delegation	Authorizing an energy manager or energy delegate to operate energy management in an effective manner.
		BI22,3 Resource provision	Preparation of the resources necessary for the operation of organizational energy management.
		BI22,4 Staff can perform efficiently in handling energy management	Proportion of employees assigned to take responsibility for the operation in order to create an impact on energy performance in the SEU area.
	Culture and behaviors [18,25,27,28,30,46,58,59]	BI23,1 Motivation and rewards	Motivational activities and rewards that support participation in the implementation and operation of energy management.
		BI23,2 Participation in energy-saving activities	Employee participation in activities associated with energy saving.
		BI23,3 Innovation for energy conservation	Innovation associated with energy conservation in an organization.
		BI23,4 Communication and internal networking	Communication and network building or participatory activities in energy management or energy conservation within an organization.
Energy performance	Energy planning [18,26,30,31,46]	BI31,1 Significant energy use(SEU)	Proportional energy consumption of SEU, compared to the total.
		BI31,2 Energy Audit program for SEUs	Energy performance auditing program of processes and machines for SEU.
		BI31,3 Energy performance improvement procedure	Procedure and criteria for energy performance improvement.
		BI31,4 Effective EnPIs	Determination of energy performance indicators (EnPIs) that appropriately and comprehensively cover significant energy consumption characteristics.
	Implementation and operation [18,30,46]	BI32,1 Preventive maintenance for SEU	Implementation of preventive maintenance associated with energy performance in the SEU area.
		BI32,2 Operational control procedure for energy saving	Implementation process for the operational control associated with energy performance.
		BI32,3 Regulatory compliance	Implementation associated with legal issues and other energy requirements applicable in an organization.
		BI32,4 Staff engagement and competence training	Competency-based training associated with energy management and performance.
		BI32,5 Design and procurement procedure	Design and purchasing process of energy in consideration of energy performance.
	Performance evaluation [18,32,46]	BI33,1 Management review	Review of the energy management system by top management.
		BI33,2 Monitoring and analysis	Monitoring and analysis of the key characteristics of organizational energy performance.
		BI33,3 Verification procedure for energy performance	Verification procedure for energy saving and effective energy performance.
Environmental	Environment impact [7,48,69]	BI41,1 Environmental standard	Bringing environmental management standards into practice in an organization.
		BI41,2 Waste management	Organizational waste management associated with energy production and consumption from waste.
		BI41,3 Resource reduction	Implementation and operation of an energy management system to minimize organizational resources.

3.2. Determination of Indicator Weights

The AHP model was employed to assess the level of significance or indicator weights that have an impact on the EnMS SI criteria and aspects [50,63,64,68]. Individual comparison matrices were collected from the questionnaire survey by 15 experts from selected ISO 50001-certified companies. The analysis results are shown in Table 5; Table 6. It can be seen that the significance or weight of the economic and organizational aspects are relevant, and they are found to have a high-level impact on EnMS sustainability, at 31% and 33%, respectively. This is followed by the energy performance aspect, at 23%, and the environmental aspect, at 13%. Considering the following analytic hierarchy, the continuous benefit is shown to be the most important economic aspect, with a weight as high as 56%. With regard to the organizational aspect, commitment is the most important factor, with a weight of 60%. As for the analytic hierarchy of the commitment criteria, significant indicators are top management commitment and long-term strategic planning, with relevant weights of 48% and 41%, respectively. In terms of the organizational criteria, culture, and behaviors, the significant indicators are embedding EnMS property into the business process and motivation and rewards, respectively. Energy planning is the most important factor of the energy performance aspect, with a weight of 55%. According to the analytic hierarchy, significant indicators in this group are the energy used, the energy audit program for SEUs, and the management review, monitoring, and analysis. In relation to the environmental aspect, the environmental standard is given the highest significance, with a weight of 55%, but considering the bigger picture, of all the indicator weights impacting on the EnMS, the most significant indicators are continuous benefits, top management commitment, and long-term strategic planning.

Table 5. Determination of the weight of the four aspects of EnMS sustainability by the AHP model.

Aspect	Statistics	Aspect	Statistics
AHP Model	Weight	AHP Model	Weight
W1, Economic	0.31
W2, Organizational	0.33
W3, Energy performance	0.23
W4, Environmental	0.13
Eigenvalue(λ)—average value	4.08
Consistency ratio (CR)	0.03
Organizational aspect	Weight	Energy performance aspect	Weight
WG21, Commitment	0.6	WG31, Energy planning	0.55
WG22, Organizing ability	0.2	WG32, Implementation and operation	0.24
WG23, Culture and behaviors	0.2	WG31, Performance evaluation	0.21
Eigenvalue(λ)—average value	3	Eigenvalue(λ)—average value	3.02
Consistency ratio (CR)	0	Consistency ratio (CR)	0.01

Table 6. Determination of the weight of the EnMS sustainability indicators by the AHP model.

Aspect	Statistics	Aspect	Statistics
Economic—investment and returns	Weight	Organizational—commitment	Weight
WB11,1,, Continuous benefit	0.56	WB21,1,Top management commitment	0.48
WB11,2,, Proportional energy cost	0.28	WB21,2, Long-term strategic planning	0.41
WB11,3,, EE project investment	0.11	WB21,3, Energy policy	0.11
WB11,4, EE investment procedure	0.06	Eigenvalue(λ)—average value	3.03
Eigenvalue(λ)—average value	4.09	Consistency ratio (CR)	0.03
Consistency ratio (CR)	0.03
Organizational—organizing	Weight	Organizational—culture and behaviors	Weight
WB22,1,, EnMS properly embedded into the business process	0.42	WB23,1, Motivation and rewards	0.46
WB22,2, Energy management and power delegation	0..22	WB23,2, Participation in energy-saving activities	0.31
WB22,3, Resource provision	0.25	WB23,3, Innovation for energy conservation	0.12
WB22,4, The efficiency of the performance of staff in handling energy management	0.11	WB23,4, Communication and internal networking	0.11
Eigenvalue(λ)—average value	4.05	Eigenvalue(λ)—average value	4.05
Consistency ratio (CR)	0.02	Consistency ratio (CR)	0.02
Energy performance—energy planning	Weight	Energy performance—implementation and operation	Weight
WB31,1, Significant energy use	0.33	WB32,1, Preventive maintenance for SEU	0.29
WB31,2, Energy audit program for SEUs	0.36	WB32,2, Operational control procedure for energy saving	0.29
WB31,3, Energy performance improvement procedure	0.16	WB32,3, Regulatory compliance	0.22
WB31,4, Effective EnPIs	0.15	WB32,4, Staff engagement and competence training	0.13
Eigenvalue(λ)—average value	4.02	WB32,5, Design and procurement procedure	0.08
Consistency ratio (CR)	0.01	Eigenvalue(λ)—average value	5.17
		Consistency ratio (CR)	0.04
Energy performance—monitoring and verification	Weight	Environmental—environment impact	Weight
WB33,1, Monitoring and analysis	0.30	WB41,1, Environmental standard	0.55
WB33,2, Verification procedure for energy performance	0.16	WB41,2, Waste management	0.24
WB33,3, Management review	0.54	WB41,3, Resource reduction	0.21
Eigenvalue(λ)—average value	3.01	Eigenvalue(λ)—average value	3.02
Consistency ratio (CR)	0.01	Consistency ratio (CR)	0.02

3.3. Survey and Assessment of Sustainability Indicators with a Sample

In order to employ the above-mentioned framework in assessing the sustainability index of each company, questionnaires were used for interviewing energy management representatives (EnMR) or other persons in companies related to energy management. The questionnaires for the qualitative assessment of basic indicators are shown in Table A1 of the Appendix A. The results obtained from workplace assessments are calculated in the EnMS SI.

The questionnaire and interview data were collected and analyzed from the end of 2017 to the beginning of 2019. The questionnaires were sent to 108 companies in Thailand that had held an ISO 50001 certification for at least one year. The assessment of qualitative score levels for each basic indicator requires the evaluator’s understanding and an appropriate amount of supportive data. As a result, a great deal of time was spent responding thoroughly to the questionnaires. The questionnaires from 31 companies were returned, accounting for 28% of the companies engaged. The questionnaires were distributed to the types of industries shown in Figure 3.

Figure 3. Proportion of companies, categorized by the type of industry.

The data obtained from the primary survey of 31 companies indicate that the main drivers of the adoption of ISO 50001 in an organization were the requirement of top management to reduce energy costs, followed by increasing the organization’s competitive advantage, and the operation being in accordance with the company’s environmental mission. This is consistent with previous research studies, in that internal drivers are shown to have a greater influence on the motivation to adopt ISO 50001 than external drivers [1]. The scores for 30 indicators, from the assessment of 31 companies, are shown in Table A2 in the Appendix A.

3.4. Index Construction

The assessment results for 30 sustainability indicators in 31 companies, together with the indicator weights obtained through the AHP, were applied to evaluate the aggregate value of sustainability indicators using the weighted arithmetic mean (WAM) method. The mean of sustainability indicators for each aspect is shown in Figure 4, and the calculation results for the aggregate value of the four aspects and the EnMS SI are shown in Table A3 in the Appendix A. The broader context of the EnMS SI for 31 companies can be more clearly seen in Figure 5.

Figure 4. The mean of the basic sustainability indicators from 31 companies.

Figure 5. Score levels for the four aspects and the EnMS SI of 31 companies.

4. Discussion

The main objective of this research was to develop a new method for assessing the quality of energy management in the form of an EnMS SI, using qualitative score levels as sustainability indicators which directly impact on the energy performance of ISO 50001-certified companies. The framework proposed here can be used as a tool to assess the quality, sustainability, and impact of an organizational EnMS, and the results were applied to achieve sustainability and optimal benefits in business operations. The quality levels of 30 sustainability indicators are shown in Figure 4, enabling consideration to be devoted to the quality of each process or the current indicators of an organizational EnMS, which are presented, in this study, in the form of mean values for 31 companies. The indicators of the economic aspect range from 1.94–2.06, which is the lowest mean, followed by the culture and behavior criteria of the organization aspect, which range from 2.23–2.35. Considering the broader context and applying the assumption that all 31 companies meet the ISO 50001 standard for factories in Thailand, it can be concluded that the weak points of the EnMS ISO 50001 standard in Thailand are internal economics and the culture and behaviors, and this is in accordance with previous studies on energy efficiency projects in other countries [28,58,59].

The results concerning the quality score levels of the activities and processes based on each indicator are shown to be comparable to the Energy Management Maturity Model (EM3) and the Energy Management Maturity Model based on ISO 50001 (EMMM50001) [36,41,42]. The assessment results, which are in accordance with EMMM50001, show an increase in the score levels during each EnMS process, based on the PDCA cycle guideline, from the EnMS establishment to the energy management review. The scores ranged from 1 to 5, depending on the advancement levels of the process (Level 1: Initial to Level 5: Optimized), provided that the levels of importance of the processes were equal [42]. However, the EnMS SI framework provides a more in-depth assessment of the quality and effectiveness of the indicators in each process of energy management and related areas (reducing the environmental impact, waste, and resources), thereby affecting organizational energy performance. From Figure 5, it can be seen that the EnMS SI obtained from the quality assessment of indicators, in conjunction with the aggregate weight values, has a significant impact on energy management sustainability, when considering the current status of each company. Seven companies have a sustainability index lower than 2.0, with company 18 exhibiting the lowest. Among these, companies 13, 15, 18, and 21 have quite a low score level in terms of the economic aspect. In contrast, nine companies have a sustainability index higher than 2.5, and company 16 has the highest. Based on the score levels for each aspect, the findings indicate that all companies in this group have a significantly high score for the economic aspect.

While the EnMS SI framework seems to have a positive bias when applied to assess energy-intensive industries, these industries are considered to have a greater need for energy management implementation than others due to the effect of cost-based competition. Conversely, in industries such as hospitality, the EnMS SI framework is designed within the context of ISO 50001-certified organizations. Therefore, these differences are mostly compensated by the certification process of energy management [43,44]. However, ISO 50001 is a process standard, not a performance standard [7], and confirms the effectiveness of successful energy management and its intended outcomes [46], for which the EnMS SI framework already proposes outcome indicators, such as continuous benefits, long-term strategic planning, participation in energy-saving activities, etc. The limitations of this study involve the deviation of results, including the understanding and knowledge of the evaluators, as well as the weak economic conditions of the organizations involved, and the fact that this framework can only be applied to ISO 50001-certified companies in Thailand.

5. Conclusions

The EnMS SI framework proposed in this study can be applied to—and confer benefits on—all organizations in both the industrial and service sectors through the self-assessment of the EnMS quality aspects using the BSI questionnaire, as shown in Table A1. The scale (1, 2, 3) of each BSI is used to evaluate the quality of the EnMS SI in each organization. In this regard, evaluators can predict the current sustainability potential of an EnMS and analyze its strengths and weaknesses. Additionally, they can employ this important information in the process of strategic decision-making for improvement and sustainability. In the future, the developed index could be promoted by constructing a decision support system (DSS) to increase the impact of sustainable energy management [70]. This framework would enhance any companies adopting ISO 50001 in Thailand, increase their number and maintain sustainability. The high quality of ISO 50001 will help organizations to achieve a continuous energy performance improvement and return the benefits to companies, interested parties, and the world. It will also reduce the environmental impact, carbon dioxide emissions, and the problems caused by climate change [7,24,61,62].

While a second addition of ISO 50001 was published in August 2018 [46], the EnMS SI framework can still be applied, since the standard requirements and PDCA process remain the same. The new edition requires that organizations implement and operate their energy management and business process consistently and apply the analyzed results to energy planning. In a case where the second edition of ISO 50001 has been applied for a while, and an organization decides to stop implementing the standard, it is essential to examine the main reasons for such a decision. This is because, in addition to the internal economic problems identified in this study, there may be other causes that have a significant effect on decision-making, and guidelines on formulating a policy to promote energy management in Thailand in the future are needed.