Sustainability Assessment in Manufacturing under a Strong Sustainability Perspective—An Ecological Neutrality Initiative

Abstract

The concept of strong sustainability establishes ecosystem conservation as the basis for socioeconomic development. Despite the increase in the number of studies on this subject, the qualitative approach used in studies on strong sustainability makes the introduction of this theme difficult in the industrial context. The absence of a model of sustainability evaluation in manufacturing based on the concept of strong sustainability was the gap identified by this research. The objective of this study was to develop a model that embeds strong sustainability within the sustainability assessment of manufacturing companies. The research used survey methodology to obtain the opinion of experts on the relevance of sustainability metrics. Information collected from experts was used to calculate the weights of indicators and of the participation of each dimension in strong sustainability. The results indicated that strong sustainability consists of 48% of environmental, 29% of social, and 23% of economic factors. The model has been applied in a study of multiple cases in factories in the automotive sector, two in Brazil and two in Italy. The results revealed that the four companies were rated regular in the strong sustainability scale. However, the sustainability performances of the companies showed different patterns over five years. Furthermore, analysis of the individual performance of the dimensions showed that the economic growth of the two Brazilian factories was superior to the socio-environmental development. The result of the Italian units emphasized different priorities. A firm reached the best result in environmental performance and the other one on the social dimension.

1. Introduction

The concern with the sustainability of industrial operations has been focused on decision-making, which are oriented to meet social demands, environmental preservation, and business profitability [1]. Sustainable products and processes increase value to the corporate image, revealing a competitive advantage for companies [2] and increased sales through increased customer satisfaction and loyalty [3].

Sustainable business management requires integration of measurement, evaluation, and result dissemination practices in order to improve the transparency and performance [4]. Sustainability evaluation systems are usually employed to reach sustainability [5]. The assessment process is conducted through the monitoring of indicators [6]. Regarding the dissemination of results, the Global Reporting Initiative (GRI) standard is the most adopted by companies to report sustainability performance [7].

In terms of sustainability assessment, there are mathematical models developed to calculate the performance of companies, with emphasis on studies conducted in the sectors of construction, agriculture and energy. Examples of this are the sustainability assessment of green buildings [8], sugarcane biofuel to support public policies [9], and sustainable solutions for energy service [10]. However, studies on sustainability assessment in the automotive sector have a qualitative approach [11] or have failed to cover the three pillars of sustainability: environmental preservation, economic growth, and social well-being [12].

The concept of Triple Bottom Line (TBL) suggests the balance of economic, environmental, and social factors in corporate decision-making [13]. This concept has influenced managerial approach and has gained notoriety in regulatory agencies, consultancies, and non-governmental organizations [14]. Although TBL is relevant to disseminating environmental and social issues in operations management, this approach is not sufficient to ensure ecosystem conservation. TBL infers equivalent weight to economic, environmental, and social dimensions as pillars of sustainable development [15], which denotes the substitutability model implicit in weak sustainability [16].

By definition, in weak sustainability manufactured capital is a good substitute for natural capital, e.g., a waste treatment plant would replace the water purification process accomplished by a forest ecosystem [17]. The thresholds of depletion of natural resources are ignored in the analysis of weak sustainability as well as the limits of substitutability between natural and manufactured capital [18]. Thus, the evaluation from the perspective of weak sustainability results in neglecting the exhaustion of natural resources.

On the other hand, the concept of strong sustainability defines that natural capital is fundamental for human well-being and economic growth [17]. This statement is supported by the fact that the availability of natural resources is limited, while manufactured products are reproducible and social demands change with each generation [19]. Thus, scarcity occurs exclusively in natural resources and the strong sustainability approach is adequate for ecosystem maintenance [20]. However, the quantification of the limits of a natural system requires deep knowledge of environmental science, making this approach challenging in the corporate context [21].

Despite the complexity, the strong sustainability approach has been considered in scientific researches that assess sustainability. Shang et al. [22] investigated progress for three decades in Mongolia and concluded that weak sustainability is not sustainable, suggesting to adopt a strong sustainability approach. In Japan, Uehara and Mineo [23] proposed a framework to manage the integrated coastal zone with the perspective of strong sustainability through the definition of threshold values and system resilience. Additionally, Shmelev and Rodrígues-Labajos [24] analyzed the performance of fifteen European countries from the perspective of strong sustainability, based on six indicators with targets defined as thresholds by the European Commission.

Furthermore, regarding the thresholds of the natural system, Janeiro and Patel [25] suggested that decision-making be made based on factors that impact climate change, reduction of biodiversity, nitrogen and phosphorous cycles, degradation of the ozone layer, acidification of oceans, availability of drinking water and soil transformation. Pollesch and Dale [26] developed a mathematical model of non-compensatory aggregation function based on strong sustainability.

Cinelli et al. [27] identified three multi-criteria decision-making methods, Elimination and Choice Expressing the Reality (ELECTRE), Preference Ranking Organization Method for Enrichment of Evaluations (PROMETHEE) and Dominance-based Rough Set Approach (DRSA), that limit or nullify the compensation between sustainability dimensions. Romero and Linares [28] identified disadvantages and complications in using exergy as an indicator of strong sustainability. Additionally, Prato [29] used fuzzy logic to evaluate ecosystem sustainability from the perspective of strong sustainability. Despite the presence of strong sustainability in global scale analyses, the manufacturing-focused studies analyzed in this research ignored the approach to this theme.

Additionally, the analysis conducted in studies on sustainability assessment in manufacturing identified flaws in the scope of the economic, environmental, and social dimensions. Thus, studies showed absence of at least one dimension in the evaluation performed. For example, eco-efficiency was the focus on evaluating improvements in production systems [30,31] in the definition of the investment strategy of Chinese manufacturing industries [32] and in the analysis of rubber processing in Sri Lanka [33]. In contrast, the social factor was the only concern when evaluating sustainability in Indian automotive companies [34], fair wage based on the life cycle assessment [35] and operational initiatives, projects and technologies in industry [36].

Another aspect to be considered is the coverage of the three manufacturing elements: product, process, and system. However, the approach of some researches was exclusively at the product level, in order to identify the impacts of product sustainability to assist manufacturing decision-making [37,38] and to integrate the life cycle concept into additive manufacturing products [39].

In addition, studies with a unique approach at the process level were found to compare dry machining to traditional processes that use cutting fluids [40], to add uncertainties from multiple sources in welding and plastic injection [41], and to propose a sustainability assessment algorithm of machining [42].

Furthermore, other studies have evaluated sustainability through the insertion of process improvement tools, such as Lean Manufacturing and Six Sigma [43], Quality Function Deployment [44,45], and Work-cell-based (SERU) production technologies [46]. Furthermore, researches identified drivers and barriers in sustainability performance through a literature review [47,48] and a case study in the Chinese metalworking industry [49].

With this, the literature analysis verified the absence of a model for evaluating sustainability performance in manufacturing with a strong sustainability approach, covering the economic, environmental, and social dimensions and the elements of manufacture, product, process and system. The challenge is to incorporate the concept of strong sustainability in sustainability assessment given that the boundaries of the ecosystem are undefined.

The observed gap in the literature suggested the following research question: How to evaluate sustainability performance in manufacturing with coverage of the three dimensions of sustainability, manufacturing elements, and strong sustainability? The central research question was divided into three specific questions: (a) Which are the key performance indicators for measuring sustainability in manufacturing? (b) Do the indicators have the same weight in the evaluation of sustainability performance from the perspective of strong sustainability? (c) How to calculate the sustainability performance of industries from the perspective of strong sustainability?

To answer these questions, this study aims to provide a quantitative approach to evaluate sustainability performance in manufacturing, covering the three dimensions of sustainability and manufacturing elements, from the perspective of strong sustainability. To achieve this general objective, five specific objectives were established: (a) Define a set of indicators to measure sustainability in manufacturing; (b) Establish the weight of each indicator from the perspective of strong sustainability; (c) Develop a mathematical model to calculate the sustainability performance of companies; (d) Collect sustainability data from industries in the Brazilian and Italian automotive sectors; (e) Evaluate the sustainability performance of companies in different countries, from the perspective of strong sustainability.

The remainder of the article is organized as follow: literature review is in Section 2; research methodology and the model proposed are in Section 3 and 4, respectively; the application of the model in a multi-case study is in Section 5, and discussion of results is in Section 6; finally, conclusions and further development are in Section 7.

2. Literature Review

2.1. Strong Sustainability

The strong sustainability concept assumes that the ecosystem is critical capital in sustainability, which implies carrying out economic activities and meeting social demands in addition to ecosystem conservation [17]. A counterpoint to this concept is weak sustainability, which allows manufactured capital to replace natural capital, e.g., the construction of water treatment plants to compensate for the endangered natural processes by deforestation and contamination of springs [17].

The compensation and substitution in the dimensions of sustainability have been widely addressed in the literature, emphasizing the difference between strong and weak sustainability. In this context, studies developed frameworks for socio-environmental labelling in products and processes [50], actions to promote strong sustainability in business operations [19], and the application of natural capital as critical to identify priorities and policies for strong sustainability [51].

Additionally, Bjørn and Røpke [21] presented methods based on the concept of ecosystem boundaries and life cycle assessment. Martins [20] identified restrictions for strong sustainability due to the predominance of the neoclassical economy. Lorek and Fuchs [52] concluded that weak sustainability is not appropriate for sustainable development challenges and proposed strong sustainability-based consumer guidelines to outline deficiencies to be addressed by policy reforms. Nilsen [53] emphasized the need for academics, governments, and managers to seek strong sustainability alternatives for global development. Dietz and Neumayer [54] pointed out the best practices in measuring strong and weak sustainability.

In studies applied in the practical field, Pelenc and Ballet [55] analyzed the relationship between human well-being and critical natural capital to demonstrate the implementation of human development projects from the perspective of strong sustainability. Bishop et al. [56] proposed a model using strong sustainability to integrate electricity generation by composite fuels, which is able to reduce dependence on imported fuels and to increase the use of renewable sources for energy generation. Garmendia et al. [57] examined the role of critical thresholds and suggested tools to improve management and fishing regulation. Baumgärtner and Quaas [58] applied strong sustainability criteria to manage livestock pasture.

2.2. Sustainability Assessment in Manufacturing

Searching in the literature for models of sustainability evaluation in manufacturing resulted in 251 articles. Despite the use of the words ‘manufacturing’ or ‘industry’, 179 studies of the 251 articles found were conducted in other sectors, such as energy generation and use, supply chain management, civil construction and urban planning, waste management and recycling, chemical, pharmaceutical and transport sectors. This finding indicated that 179 articles were outside the scope of this research.

The first screening in the articles resulted in 72 studies that addressed sustainability assessment in manufacturing companies. The content analysis pointed out seven articles that proposed models to evaluate sustainability performance, covering the economic, environmental and social aspects, including product, process, and system. The other 65 studies failed at least one of the criteria mentioned above.

A group of 25 studies is outside the scope of this research due to the absence of a sustainability assessment model. Case studies have identified factors that influence sustainability performance, drivers and barriers in companies in China [49], Indonesia [59], Taiwan [60], and the United Kingdom [61]. Additionally, other cases have shown sustainability practices in Lean Six Sigma projects [43], the correlation between Triple Bottom Line and Balanced Scorecard [62], the advantages of Lean Cleaner Production [63], and the benefits of Green and Lean in small and medium-sized Indian companies [64].

Additionally, quantitative studies applied Quality Function Deployment (QFD) to define priorities for maintenance activities [44] and classify key performance indicators in manufacturing [45]. Furthermore, sustainability indicators were integrated into Lean Manufacturing to evaluate operational processes [65]. Surveys revealed sustainable business practices of small and medium companies [66], the evolutionary path of sustainable operations [67] and the parameters in the life cycle analysis of vehicles from the perspectives of processing and consumption [12]. Moreover, sustainable manufacturing drivers and barriers have been identified in Italy [47,48] and India [68].

Through the literature review, theoretical-conceptual studies identified the most widely used sustainable manufacturing indicators [69], the goals that support the development of sustainable products [70] and the aspects taken into account in decision-making of the plant installation site [71]. Furthermore, an analysis of the effect on sustainability dimensions was performed by the adoption of Lean and Green philosophy [72], and the Japanese production system SERU [46]. Additionally, a framework was developed with guidelines for sustainability assessment [73], an array of technical terms used to describe production processes [74], and the inventory of research on sustainability assessment in manufacturing [75].

Another aspect observed was the singular approach in eco-efficiency, neglecting the social aspect, which resulted in 23 articles misaligned with this research. The Greenhouse Gases (GHG) level was the objective to investigate the cement industry [76,77], the new welding and turning technologies [78], iron and steel industry [79], the supply chain in the consumer goods industry [80], and an eco-industrial park [81]. In terms of energy management, research has proposed methods to increase energy efficiency in product deliveries [82], in light and heavy industries [32] in manufacturing industry departments [83], in plastic welding and injection processes [41], in the combined use of grid electricity and internal generation [84], and in the definition of the thickness of material removed in the machining process [85].

In addition, environmental impact mitigation was the purpose to calculate the mass balance in automotive parts sectors [86], natural rubber [33], steel industry [87], cement industry [88], chemical industry [89,90], consumer goods, packaging, food and metalworking [31], particleboards [91], and pulp and paper [92]. Furthermore, Bai and Sarkis [93] applied a model based on flexibility and environment criteria to evaluate advanced manufacturing technologies in three global manufacturing companies and González-Mejía and Ma [94] used emerging flow dynamics to assess Puerto Rico’s sustainability over the transition period from an agricultural to an industrial-based economy.

Studies on sustainability assessment that focused on a single process were left out of this research. In this case, 12 sustainability performance investigations used specific metrics to types of operational processes. It means that the use of those metrics cannot be extended to all manufacturing industry. The multi-criteria approach to sustainability supported decision-making to choose advanced manufacturing technologies [95], the best product design for gear manufacturing [37], the location of a new chemical plant [96], and machining alternatives to conventional processes [97]. Furthermore, the sustainability approach was present in the analysis of manufacturing processes that evaluated cutting parameters in hard turning stainless steel [98], the life cycle stages from pre-manufacture to post-use [42], the surface integrity characteristics in machining of Inconel 718 [99], the economic, environmental and social impacts on work cells [100], sustainability in the manufacture of stamped parts [101], the effects of modular machines, auxiliary units and machine codes [102], and tool-life performance measurement [40]. In the theoretical sphere, Ulutan and Ozel [103] identified the main problems induced by machining of titanium alloys and nickel-based alloys through a literature review.

Studies dedicated exclusively to the social dimension were also left out of this research. The negligence of five investigations on economic and environmental factors denoted the failure of the models to carry out a complete sustainability assessment. In this group, Neugebauer et al. [35] calculated fair wages throughout the product life cycle. Furthermore, Rajak and Vinodh [34] applied fuzzy logic to evaluate social sustainability performance. Moreover, Labuschagne and Brent [36] built a framework with social criteria to evaluate sustainability performance of process and product technologies. Additionally, the impacts of Green Human Resources Management practices in industries in India [104] and Palestine [105] were evaluated.

Seven studies developed models to evaluate sustainability performance in manufacturing. The application of the models in corporate practice was conducted through five case studies and two surveys. Three case studies evaluated sustainability in automotive companies [38,106,107]. Furthermore, a sustainability assessment model was applied in a health products manufacturing company [108] and another in the Chemical sector [109]. The surveys were conducted in companies in the textile, food and health sectors in Mauritius [110], and in twenty small and medium-sized enterprises in Taiwan [111]. Information on the seven selected studies are in Figure 1.

The construction of the seven models identified in the literature began with the definition of indicators to measure sustainability performance. Then, methods were found to consolidate data of different dimensions into a single indicator. Subsequently, weights were assigned for each indicator according to its relevance to sustainability. Finally, the consolidation of weights and indicators for the calculation of the sustainability index of industries were carried out.

The investigation of Jiang et al. [106] measured economic results by sales, billing, operating cost, profit, research and development, market share, and global equipment effectiveness metrics. In environmental terms, energy and water consumption, conservation of natural habitat, emission of polluting gases, industrial effluent, and solid waste were monitored. The social dimension was measured by Jiang et al. [106] for job satisfaction, number of accidents, employee training, gender equity, investment in social projects, and customer satisfaction.

Sabaghi et al. [107] used operating costs to measure economic outcome. In environmental terms, the indicators were consumption of non-renewable and renewable materials, life cycle design, hazardous materials, recycling, reuse, energy, water, emission of polluting gases, industrial effluent, and solid waste. In the social aspect, Sabaghi et al. [107] measured the number of accidents.

In the study by Ghadimi et al. [38], operating cost was assumed as the economic indicator. Environmental monitoring was conducted in the consumption of non-renewable, renewable and hazardous materials, emission of polluting gases, industrial effluent, and solid waste. In the social dimension, Ghadimi et al. [38] counted the number of accidents.

The investigation of [108] used operating costs as an economic measure. Environmental performance was measured by the consumption of non-renewable and renewable materials, life cycle design, hazardous materials, recycling, reuse, energy and water consumption, emission of polluting gases, industrial effluent, and solid waste. Social factors were suggestion of improvements, job satisfaction, lost working days, number of accidents, employee training and customer satisfaction.

Krajnc and Glavic [109] took into account sales, profit and investment in research and development as economic indicators. The environmental variables were consumption of non-renewable, renewable and hazardous materials, recycling, energy and water consumption, emission of polluting gases, industrial effluent and solid waste were the metrics used. In the social dimension, Krajnc and Glavic [109] took into account number of employees, suggestion of improvements, number of accidents, and investments in social projects.

After defining sustainability indicators, the next step was to establish the data processing method to convert them into dimensional values. Sustainability indicators are measured in different quantities. The seven investigations used the normalization process to convert economic, environmental, and social data into dimensional values.

The further step was the definition of weights for economic, environmental, and social indicators. The knowledge of experts from various segments was used to construct weighted models of sustainability through multiple linear regression [110] and Gray Relational Analysis [111]. In case of studies in the automotive sector, corporate sustainability performances were calculated by Principal Component Analysis [106] and the Fuzzy Analytical Hierarchical Process [38,107]. Additionally, the Analytical Network Process was used to elaborate the model that evaluated the sustainability of the medical products factory [108] and the Analytical Hierarchical Process was the method applied to weigh the economic, environmental, and social indicators in chemical industry evaluation [109].

It is important to emphasize that the seven selected models did not consider strong sustainability in the evaluation of sustainability in manufacturing. As a result, references to the mentioned models was limited to the identification of economic, environmental, and social indicators used in manufacturing companies, the method of data collection for weighting sustainability metrics, and the methodology steps for the construction of the model.

3. Research Methodology

The strong sustainability evaluation was initiated with the systematic review of articles for the identification of performance indicators in manufacturing and the structure of the procedures used. Content analysis is documentary and infers knowledge by coding and categorizing data to select the appropriate articles in the development of the conceptual model [112].

A set of keywords was defined, and their combinations were used in searches in title, abstract and keyword, in the following databases: ScienceDirect, Scielo, Scopus, Taylor & Francis, Web of Science and Willey. The keywords used were: ‘sustainability’; ‘sustainable development’; ‘assessment’; evaluation’; ‘model’; ‘framework’; ‘manufacturing’; ‘industry’. The use of keywords aimed to seek the literature relevant to the purpose of the study [113].

The search in databases for models of sustainability evaluation in manufacturing resulted in 251 articles. The first screening was to identify studies on industry. This analysis showed 179 articles outside the scope of this research due to approach other sectors such as energy generation and use, supply chain management, civil construction and urban planning, waste management and recycling, chemical, pharmaceutical and transport sectors. Then, 72 studies addressed sustainability assessment in manufacturing companies. The second screening aimed to identify sustainability models and the approach of the three sustainability factors. This phase required to read the entire text of 72 articles. The analysis revealed 65 of 72 papers that did not fit into the purpose of this research. The reasons were the absence of a sustainability assessment model (25 articles) and the singular approach in eco-efficiency (23 papers) or social dimension (5 papers). Moreover, 12 studies presented models with specific metrics related to operations such as welding, machining and stamping. Those metrics would not be appropriated for the wide range of operations in manufacturing systems as proposed by this research.

The selection of articles on sustainability assessment in manufacturing, with the three dimensions of sustainability and manufacturing elements, resulted in seven studies that were references to structure the stages of construction of the sustainability evaluation model proposed in this research.

After the construction of the theoretical basis, a survey was conducted with specialists in strong sustainability, obtaining judgment on the relevance of sustainability metrics in manufacturing identified in the literature. A survey consists of collecting and analyzing information from individuals using research instruments [114]. The criterion adopted to identify strong sustainability experts was the selection authors of articles with the term “Strong Sustainability” in title, keywords or abstract.

The field research instrument used to obtain information was a structured questionnaire, with individual questions for each sustainability indicator. The structured questionnaire is an appropriate method for data collection in a survey [115]. This research attained 58 answered questionnaires. The response possibilities took into account the relevance of the indicator for strong sustainability: (1) low relevance; (2) medium-low relevance; (3) medium relevance; (4) medium-high relevance and; (5) high relevance. The scale with five levels of relevance for sustainability indicators was based on the method adopted by Beekaroo et al. [110], which had the participation of 30 specialists in operations management.

The application of the sustainability assessment model was conducted in four companies in the automotive sector, two in Brazil and two in Italy. The case study method has been consistently used in operations management research [116]. Case studies enable a holistic view of the routine, which emphasizes its empirical character in the investigation of the contemporary phenomenon [117]. This study presented was a descriptive research. Descriptive case study is useful for many purposes such as presenting an unusual situation to researchers [117]. The selection of the cases in a multiple-case study should follow a replication rather than sampling logic [117]. The automotive industry is one of the largest manufacturing sectors worldwide [118]. It has a long value chain and makes influence other business sectors [119]. The relevance of automotive industry denoted a suitable field to validate the model proposed by this research.

The selection of cases took into account two actions [120]. The first was setting boundaries of the study by means of choosing automotive firms that disclose sustainability reports, which reveals concerns on environmental conservation and social demands. The second action was creating a sample frame to help uncover the basic process. This research built this frame by comparison of industries in a developed country and under development one, with different incentives and barriers to promote strong sustainability in manufacturing. Firms with operations in Brazil and Italy were defined due to the convenience and importance of automotive industry in wealth generation in those countries. Convenience sampling is usually selected when the case companies are located close to the researchers [121]. The automotive chain contributes a significant share of the Brazilian [122] and Italian [123] industrial GDP.

Data collection was performed through interviews with managers and technicians, analysis of reports and observations of processes. The combination of different data collection methods is a positive aspect of the case study, which enables the investigator to analyze the studied phenomenon deeply [116]. The data collected referred to economic, environmental, and social results of companies in the period from 2014 to 2018. The period covered by this research intended to show the recent scenario of sustainable performance of the cases and compare their trends. A five-year period also was used in the sustainability evaluation model for manufacturing systems [62], in the definition of sustainability indicators of iron and steel production [124] and Russian oil and gas industry [125].

4. Model for Performance Assessment under Strong Sustainability Perspective

The definition of the sustainability assessment model in manufacturing consisted of four stages. Initially, sustainability performance indicators were defined. Subsequently, strong sustainability experts were consulted to point out the relevance of the indicators for strong sustainability. In the third stage, the ratings assigned by the experts were treated by paired comparison to calculate the weights of the metrics, from the perspective of strong sustainability. Then, the indicators and their respective weights were consolidated in the equation that calculates the sustainability index in manufacturing.

The steps of building the model of this study were referenced in procedures used for seven models of evaluation of sustainability in manufacturing, which were identified in the systematic review of the literature. The models evaluated the sustainability of textile companies and food in Mauritius [110] and chemical industry in Slovenia [109]. In Taiwan, sustainability of twenty small and medium-sized enterprises of various segments [111] and medical products factory [108] were evaluated. Additionally, the automotive sector was a focus on case studies in China [106], in Canada [107] and Iran [38]. The steps for the model construction are illustrated in Figure 2.

4.1. Selection and Impact of Sustainability Indicators

The sustainability indicators used in this research were obtained through a systematic literature review described in the Section 2.2. The set of indicators showed in Figure 1 addresses metrics that are influenced by internal and external factors to the company, composed of seven economic, fourteen environmental and eleven social parameters.

Sustainability performance is positively or negatively influenced by the results of the selected indicators. For this reason, the impact of each indicator on global performance was determined. The procedure for analyzing the impact of indicators was established in the study by [106].

The economic result improves with increased sales, revenue, profit, research and development, market share, and Overall Equipment Effectiveness (OEE). However, rising operating costs reduce economic gain, which denotes the negative impact of this metric. Regarding environmental indicators, the higher the value of seven indicators, the global result improves. In this group are the metrics of consumption of renewable materials, project for life cycle, recycling, reuse, conservation of natural habitat and compliance with environmental legislation. On the other hand, there are seven indicators that raise the global result when their values are reduced. In this case are the metrics of consumption of non-renewable materials, hazardous materials, energy, water, GHG emission, industrial effluent and solid waste.

In the social aspect, the indicators improve the global result the higher their values, except the metrics related to occupational safety. In this sense, lost working days and number of accidents should be reduced to increase social performance. The impact judgment of the indicators was conducted through consultation with three specialists in the area of operations management. The result is shown in Figure 3. The positive impact is symbolized by an arrow indicated upwards, while the negative impact indicators are down arrow.

4.2. Normalization

The data of sustainability indicators are measured in several units of measure, which makes it difficult to analyze and compare results [126]. In this sense, the normalization of the information collected was the means used in this study to consolidate the economic, environmental, and social results in a global performance indicator of sustainability. Normalization is a method used to convert data from different units of measure into dimensional values [73].

Data normalization requires considering two possible conditions of indicator outcomes [38,106,108,109,111]. The first condition occurs when performance is directly proportional to the measured value. For example, increasing profit improves economic performance, increased amount of reused material improves environmental performance, and increased job satisfaction improves social performance. Thus, the normalization of metric values directly proportional to performance was performed using Equation (1).

$$x_{ij}^{\ast }=\frac{x_{ij} - min\left(x_i\right)}{max\left(x_i\right) -min\left(x_i\right)}$$

(1)

In which:

• $x_{ij}^{\ast } \mathrm{is} \mathrm{the} \mathrm{normalized} \mathrm{value}$
• $x_{ij} \mathrm{is} \mathrm{the} \mathrm{measured} \mathrm{absolute} \mathrm{value}$
• $min\left(x_i\right) \mathrm{is} \mathrm{the} \mathrm{lowest} \mathrm{value} \mathrm{of} \mathrm{the} \mathrm{indicator} x_i$
• $max\left(x_i\right) \mathrm{is} \mathrm{the} \mathrm{highest} \mathrm{value} \mathrm{of} \mathrm{the} \mathrm{indicator} x_i$

On the other hand, some indicators reveal results inversely proportional to performance, e.g., the increase in operating costs decreases economic performance; increased energy consumption reduces environmental performance and; the increase in accidents decreases social performance. So, to normalize data in situations where the lower value results in better performance, Equation (2) was used.

$$x_{ij}^{\ast }=1-\left(\frac{x_{ij} - min\left(x_i\right)}{max\left(x_i\right) -min\left(x_i\right)}\right)$$

(2)

In which:

• $x_{ij}^{\ast } \mathrm{is} \mathrm{the} \mathrm{normalized} \mathrm{value}$
• $x_{ij} \mathrm{is} \mathrm{the} \mathrm{measured} \mathrm{absolute} \mathrm{value}$
• $min\left(x_i\right) \mathrm{is} \mathrm{the} \mathrm{lowest} \mathrm{value} \mathrm{of} \mathrm{the} \mathrm{indicator} x_i$
• $max\left(x_i\right) \mathrm{is} \mathrm{the} \mathrm{highest} \mathrm{value} \mathrm{of} \mathrm{the} \mathrm{indicator} x_i$

4.3. Weighting of Indicators

The weighting of indicators played an important role in valuing the most relevant metrics for strong sustainability. The allocation of weights to indicators is a qualitative process by nature [127]. Usually, the weighting of indicators is performed based on the experience of specialists in the field, survey or multi-criteria decision methods [73].

At this stage, experts were consulted to indicate the relevance of economic, environmental and social metrics for strong sustainability. The answers of the experts are presented in

Table A1. Experts’ judgments on relevance of indicators in the strong sustainability perspective.

Indicator	Expert 1	Expert 2	Expert 3	Expert 4	Expert 5	Expert 6	Expert 7	Expert 8	Expert 9	Expert 10	Expert 11	Expert 12	Expert 13	Expert 14	Expert 15	Expert 16	Expert 17	Expert 18	Expert 19	Expert 20	Expert 21	Expert 22	Expert 23	Expert 24	Expert 25	Expert 26	Expert 27	Expert 28	Expert 29	Expert 30	Expert 31	Expert 32	Expert 33	Expert 34	Expert 35	Expert 36	Expert 37	Expert 38	Expert 39	Expert 40	Expert 41	Expert 42	Expert 43	Expert 44	Expert 45	Expert 46	Expert 47	Expert 48	Expert 49	Expert 50	Expert 51	Expert 52	Expert 53	Expert 54	Expert 55	Expert 56	Expert 57	Expert 58
E01	3	5	1	1	2	3	1	3	3	2	4	4	3	3	2	3	1	2	5	2	1	5	4	3	4	3	2	3	4	2	3	5	3	5	5	2	4	3	4	3	3	5	3	2	1	1	3	5	3	3	1	4	2	3	4	4	3	2
E02	3	5	1	1	2	3	3	3	3	2	5	4	4	4	3	3	1	2	5	2	1	5	3	3	3	3	2	4	5	3	4	5	3	4	5	1	4	3	5	4	3	4	3	5	2	1	3	4	3	3	1	3	4	3	4	3	3	2
E03	3	5	1	1	2	3	1	3	3	2	3	3	5	5	4	5	1	3	5	4	1	5	3	5	3	3	3	4	4	3	5	5	3	4	5	1	3	3	5	5	5	4	3	4	2	1	3	5	3	2	1	3	2	3	3	3	3	2
E04	4	5	3	1	2	3	1	4	3	3	3	4	5	5	3	1	1	1	4	2	1	4	4	2	4	4	4	5	5	2	5	4	4	4	4	2	4	3	3	5	3	4	4	5	2	1	4	4	3	3	2	4	3	3	4	4	4	2
E05	3	5	1	1	2	3	4	3	3	2	5	4	2	2	4	5	1	5	5	1	1	5	3	5	1	3	3	4	5	4	2	5	3	4	5	2	3	3	5	2	5	4	3	4	1	1	3	4	3	3	1	1	4	3	3	1	3	2
E06	3	5	3	1	2	3	3	2	3	2	2	5	2	2	2	1	1	1	3	1	1	3	4	2	1	2	2	2	4	1	2	5	2	3	3	4	1	1	3	2	3	3	2	2	3	1	2	4	3	2	1	1	2	3	1	1	2	2
E07	3	5	4	1	1	4	1	3	3	4	4	5	4	4	3	3	1	4	3	3	3	3	5	3	4	3	4	2	5	1	4	5	3	3	3	3	3	3	4	4	3	3	3	4	3	3	3	5	3	4	2	4	4	3	3	4	3	1
N01	5	5	5	5	4	5	5	3	5	5	5	4	4	4	3	5	5	5	5	5	5	5	5	5	5	3	5	5	5	4	4	5	5	5	5	5	4	4	4	4	5	5	3	5	5	5	3	5	5	3	5	5	5	5	4	5	3	4
N02	5	5	5	5	4	5	3	5	5	5	5	4	3	3	4	5	3	5	5	5	3	5	4	5	3	5	4	3	5	4	3	5	5	5	5	5	3	4	4	3	5	5	5	5	5	5	5	5	5	3	5	3	4	5	3	3	5	4
N03	4	5	5	5	5	5	5	5	4	5	3	5	4	4	5	3	3	4	5	4	3	5	5	3	5	5	4	3	5	5	4	5	5	5	5	4	5	5	5	4	3	5	5	5	4	5	5	4	5	5	3	5	5	5	5	5	5	5
N04	5	5	3	3	4	5	4	3	5	5	3	5	5	5	3	5	5	5	5	4	3	5	4	5	3	3	4	4	4	4	5	5	5	4	5	4	5	4	3	5	5	4	3	5	5	5	3	4	5	4	4	3	4	5	5	3	3	4
N05	4	5	5	3	4	5	5	4	4	4	5	4	5	5	4	5	5	5	5	3	3	5	5	5	4	4	3	4	5	5	5	4	4	5	5	4	4	4	4	5	5	5	4	3	3	5	4	5	5	4	4	4	5	5	4	4	4	4
N06	4	5	5	3	5	5	5	5	3	5	5	4	4	4	4	5	5	5	5	4	3	5	5	5	4	5	4	4	5	5	4	5	4	4	5	3	4	4	4	4	5	4	5	4	4	5	5	5	5	4	4	4	5	5	4	4	5	5
N07	5	5	5	3	4	5	3	5	3	5	5	4	5	5	3	5	5	5	5	5	3	5	5	5	2	5	5	2	5	5	5	5	5	4	5	3	3	5	3	5	5	4	5	5	5	5	5	5	5	5	4	2	5	5	3	2	5	4
N08	5	5	5	5	3	5	3	5	3	5	5	5	4	4	3	3	3	5	5	4	5	5	5	5	4	5	5	1	5	4	4	4	4	3	5	3	3	5	4	4	5	3	5	4	5	5	5	5	5	3	4	4	4	5	3	4	5	3
N09	3	5	5	5	5	5	2	5	5	5	5	4	4	4	3	4	4	5	5	4	5	5	4	5	5	5	5	4	3	5	4	1	5	4	5	5	5	3	5	4	5	4	5	3	4	5	5	4	5	5	5	5	4	5	5	5	5	5
N10	5	5	4	5	5	5	5	5	3	5	5	5	4	4	3	4	4	5	5	5	5	5	5	5	3	5	3	5	5	4	4	1	5	3	5	4	4	4	5	4	5	3	5	5	5	5	5	5	5	5	4	3	4	5	4	3	5	5
N11	5	5	5	3	4	5	3	4	4	5	4	4	4	4	3	5	5	5	5	3	3	5	5	5	4	4	4	3	5	5	4	2	4	3	5	4	5	4	3	4	5	3	4	5	3	5	4	4	5	3	4	4	4	5	5	4	4	4
N12	5	5	5	3	4	5	3	5	4	5	3	4	5	5	4	5	5	5	5	2	3	5	5	5	4	5	3	3	5	4	5	3	4	5	5	4	3	4	3	5	5	5	5	3	4	5	5	5	5	3	4	4	4	5	3	4	5	4
N13	4	5	3	3	3	4	5	5	3	3	5	4	5	5	5	3	3	3	5	4	3	5	3	3	3	5	4	2	3	3	5	1	5	5	5	5	5	3	5	5	5	5	5	5	2	3	5	3	5	5	3	3	3	5	5	3	5	3
N14	5	5	3	5	4	5	3	5	3	5	3	5	4	4	5	3	3	3	5	3	3	5	2	3	3	5	4	5	5	4	4	1	4	5	5	4	1	3	3	4	5	5	5	4	2	3	5	5	5	4	3	3	3	5	1	3	5	4
S01	1	3	1	3	1	3	1	1	1	2	1	3	5	5	4	1	1	2	4	1	3	4	4	2	1	1	3	3	3	2	5	3	3	1	4	2	1	2	4	5	1	1	1	1	1	2	1	5	1	3	1	1	2	1	1	1	1	1
S02	3	5	3	3	3	3	5	2	4	3	3	4	4	4	4	3	3	5	4	3	3	4	4	5	2	2	4	4	5	5	4	3	4	2	4	2	5	2	3	4	5	2	2	4	1	5	2	4	5	4	2	2	3	5	5	2	2	3
S03	4	5	3	3	4	3	1	3	2	5	4	5	4	4	5	3	3	2	5	4	3	5	5	2	3	3	4	5	4	3	4	3	4	3	5	3	3	2	4	4	5	3	3	5	1	2	3	5	5	5	1	3	4	5	3	3	3	4
S04	3	5	3	3	2	4	3	3	1	2	4	4	3	3	4	5	1	5	4	3	3	4	3	5	3	3	3	3	4	3	3	1	3	3	4	1	3	2	4	3	3	3	3	5	1	1	3	4	3	4	1	3	3	3	3	3	3	2
S05	5	5	3	3	4	4	5	3	1	5	5	5	5	5	4	5	3	5	5	5	3	5	4	5	3	3	5	4	5	5	5	4	4	5	5	2	3	2	5	5	5	5	3	5	4	5	3	4	5	5	1	3	3	5	3	3	3	4
S06	4	5	3	3	1	4	2	4	5	5	3	5	5	5	4	1	1	3	5	3	3	5	5	3	3	4	3	4	4	4	5	2	4	4	5	4	5	2	5	5	4	4	4	5	3	3	4	4	4	4	2	3	3	4	5	3	4	1
S07	5	5	3	3	3	3	4	3	1	5	3	5	5	5	2	1	1	3	5	5	3	5	4	3	2	3	4	5	5	5	5	2	4	3	5	1	3	2	3	5	3	3	3	5	4	3	3	5	3	4	1	2	3	3	3	2	3	3
S08	3	5	3	3	4	4	4	2	3	5	3	4	3	3	2	1	1	4	5	4	3	5	3	4	4	2	4	4	5	3	3	1	3	3	5	1	2	2	2	3	3	3	2	3	2	4	2	4	3	5	2	4	3	3	2	4	2	4
S09	5	5	3	3	4	5	4	3	2	5	3	4	5	5	4	1	1	4	4	3	3	4	4	4	3	3	5	5	5	5	5	1	4	5	4	2	2	2	3	5	5	5	3	5	4	4	3	4	5	4	2	3	4	5	2	3	3	4
S10	2	5	1	3	3	3	1	4	5	3	4	4	4	4	5	1	1	2	3	4	3	3	3	2	3	4	3	4	5	4	4	2	4	5	3	3	4	1	5	4	3	5	4	5	2	2	4	4	3	5	2	3	4	3	4	3	4	3
S11	5	5	3	3	4	4	5	3	2	3	5	4	5	5	4	1	1	4	5	5	3	5	3	4	3	3	5	2	4	4	5	1	5	5	5	4	4	3	5	5	5	5	3	5	1	4	3	4	5	5	2	3	4	5	4	3	3	4

. The next step was the paired comparison analysis. The paired comparison analysis of expert information was noted in the studies by [38,107,108,109,110]. This procedure was necessary to obtain the intensity of importance between indicators. The method used a set of paired comparison matrices with a scale of numbers that indicate how many times an element is more dominant or important over another element. The paired comparison scale is shown in

Table 1. Scale of paired comparison of indicators. Source: Adapted from Saaty and Vargas [128].

Factor of Preference, p	Importance Definition
1	Equal importance
3	Moderate importance of one over another
5	Strong or essential importance of one over another
7	Very strong or demonstrated importance of one over another
9	Extreme importance of one over another
2, 4, 6, 8	Intermediate values
Reciprocal, 1/p	Reciprocal for inverse comparison

.

Thus, 58 quadratic matrices were constructed, equivalent to the number of forms answered by specialists, with 32 rows and columns referring to sustainability indicators. The comparison matrix presented in

Table A2. Pairwise comparison matrix of an expert’s judgments.

Ind.		E01	E02	E03	E04	E05	E06	E07	N01	N02	N03	N04	N05	N06	N07	N08	N09	N10	N11	N12	N13	N14	S01	S02	S03	S04	S05	S06	S07	S08	S09	S10	S11
	Jud.	3	3	3	3	4	3	3	5	5	4	5	4	4	5	5	3	5	5	5	4	5	1	3	4	3	5	4	5	3	5	2	5	Total
E01	3	1	1	1	1	1/3	1	1	1/5	1/5	1/3	1/5	1/3	1/3	1/5	1/5	1	1/5	1/5	1/5	1/3	1/5	5	1	1/3	1	1/5	1/3	1/5	1	1/5	3	1/5	21.9
E02	3	1	1	1	1	1/3	1	1	1/5	1/5	1/3	1/5	1/3	1/3	1/5	1/5	1	1/5	1/5	1/5	1/3	1/5	5	1	1/3	1	1/5	1/3	1/5	1	1/5	3	1/5	21.9
E03	3	1	1	1	1	1/3	1	1	1/5	1/5	1/3	1/5	1/3	1/3	1/5	1/5	1	1/5	1/5	1/5	1/3	1/5	5	1	1/3	1	1/5	1/3	1/5	1	1/5	3	1/5	21.9
E04	3	1	1	1	1	1/3	1	1	1/5	1/5	1/3	1/5	1/3	1/3	1/5	1/5	1	1/5	1/5	1/5	1/3	1/5	5	1	1/3	1	1/5	1/3	1/5	1	1/5	3	1/5	21.9
E05	4	3	3	3	3	1	3	3	1/3	1/3	1	1/3	1	1	1/3	1/3	3	1/3	1/3	1/3	1	1/3	7	3	1	3	1/3	1	1/3	3	1/3	5	1/3	52.3
E06	3	1	1	1	1	1/3	1	1	1/5	1/5	1/3	1/5	1/3	1/3	1/5	1/5	1	1/5	1/5	1/5	1/3	1/5	5	1	1/3	1	1/5	1/3	1/5	1	1/5	3	1/5	21.9
E07	3	1	1	1	1	1/3	1	1	1/5	1/5	1/3	1/5	1/3	1/3	1/5	1/5	1	1/5	1/5	1/5	1/3	1/5	5	1	1/3	1	1/5	1/3	1/5	1	1/5	3	1/5	21.9
N01	5	5	5	5	5	3	5	5	1	1	3	1	3	3	1	1	5	1	1	1	3	1	9	5	3	5	1	3	1	5	1	7	1	99.0
N02	5	5	5	5	5	3	5	5	1	1	3	1	3	3	1	1	5	1	1	1	3	1	9	5	3	5	1	3	1	5	1	7	1	99.0
N03	4	3	3	3	3	1	3	3	1/3	1/3	1	1/3	1	1	1/3	1/3	3	1/3	1/3	1/3	1	1/3	7	3	1	3	1/3	1	1/3	3	1/3	5	1/3	52.3
N04	5	5	5	5	5	3	5	5	1	1	3	1	3	3	1	1	5	1	1	1	3	1	9	5	3	5	1	3	1	5	1	7	1	99.0
N05	4	3	3	3	3	1	3	3	1/3	1/3	1	1/3	1	1	1/3	1/3	3	1/3	1/3	1/3	1	1/3	7	3	1	3	1/3	1	1/3	3	1/3	5	1/3	52.3
N06	4	3	3	3	3	1	3	3	1/3	1/3	1	1/3	1	1	1/3	1/3	3	1/3	1/3	1/3	1	1/3	7	3	1	3	1/3	1	1/3	3	1/3	5	1/3	52.3
N07	5	5	5	5	5	3	5	5	1	1	3	1	3	3	1	1	5	1	1	1	3	1	9	5	3	5	1	3	1	5	1	7	1	99.0
N08	5	5	5	5	5	3	5	5	1	1	3	1	3	3	1	1	5	1	1	1	3	1	9	5	3	5	1	3	1	5	1	7	1	99.0
N09	3	1	1	1	1	1/3	1	1	1/5	1/5	1/3	1/5	1/3	1/3	1/5	1/5	1	1/5	1/5	1/5	1/3	1/5	5	1	1/3	1	1/5	1/3	1/5	1	1/5	3	1/5	21.9
N10	5	5	5	5	5	3	5	5	1	1	3	1	3	3	1	1	5	1	1	1	3	1	9	5	3	5	1	3	1	5	1	7	1	99.0
N11	5	5	5	5	5	3	5	5	1	1	3	1	3	3	1	1	5	1	1	1	3	1	9	5	3	5	1	3	1	5	1	7	1	99.0
N12	5	5	5	5	5	3	5	5	1	1	3	1	3	3	1	1	5	1	1	1	3	1	9	5	3	5	1	3	1	5	1	7	1	99.0
N13	4	3	3	3	3	1	3	3	1/3	1/3	1	1/3	1	1	1/3	1/3	3	1/3	1/3	1/3	1	1/3	7	3	1	3	1/3	1	1/3	3	1/3	5	1/3	52.3
N14	5	5	5	5	5	3	5	5	1	1	3	1	3	3	1	1	5	1	1	1	3	1	9	5	3	5	1	3	1	5	1	7	1	99.0
S01	1	1/5	1/5	1/5	1/5	1/7	1/5	1/5	1/9	1/9	1/7	1/9	1/7	1/7	1/9	1/9	1/5	1/9	1/9	1/9	1/7	1/9	1	1/5	1/7	1/5	1/9	1/7	1/9	1/5	1/9	1/3	1/9	4.8
S02	3	1	1	1	1	1/3	1	1	1/5	1/5	1/3	1/5	1/3	1/3	1/5	1/5	1	1/5	1/5	1/5	1/3	1/5	5	1	1/3	1	1/5	1/3	1/5	1	1/5	3	1/5	21.9
S03	4	3	3	3	3	1	3	3	1/3	1/3	1	1/3	1	1	1/3	1/3	3	1/3	1/3	1/3	1	1/3	7	3	1	3	1/3	1	1/3	3	1/3	5	1/3	52.3
S04	3	1	1	1	1	1/3	1	1	1/5	1/5	1/3	1/5	1/3	1/3	1/5	1/5	1	1/5	1/5	1/5	1/3	1/5	5	1	1/3	1	1/5	1/3	1/5	1	1/5	3	1/5	21.9
S05	5	5	5	5	5	3	5	5	1	1	3	1	3	3	1	1	5	1	1	1	3	1	9	5	3	5	1	3	1	5	1	7	1	99.0
S06	4	3	3	3	3	1	3	3	1/3	1/3	1	1/3	1	1	1/3	1/3	3	1/3	1/3	1/3	1	1/3	7	3	1	3	1/3	1	1/3	3	1/3	5	1/3	52.3
S07	5	5	5	5	5	3	5	5	1	1	3	1	3	3	1	1	5	1	1	1	3	1	9	5	3	5	1	3	1	5	1	7	1	99.0
S08	3	1	1	1	1	1/3	1	1	1/5	1/5	1/3	1/5	1/3	1/3	1/5	1/5	1	1/5	1/5	1/5	1/3	1/5	5	1	1/3	1	1/5	1/3	1/5	1	1/5	3	1/5	21.9
S09	5	5	5	5	5	3	5	5	1	1	3	1	3	3	1	1	5	1	1	1	3	1	9	5	3	5	1	3	1	5	1	7	1	99.0
S10	2	1/3	1/3	1/3	1/3	1/5	1/3	1/3	1/7	1/7	1/5	1/7	1/5	1/5	1/7	1/7	1/3	1/7	1/7	1/7	1/5	1/7	3	1/3	1/5	1/3	1/7	1/5	1/7	1/3	1/7	1	1/7	9.6
S11	5	5	5	5	5	3	5	5	1	1	3	1	3	3	1	1	5	1	1	1	3	1	9	5	3	5	1	3	1	5	1	7	1	99.0

illustrates this procedure. The presentation of the 58 matrices is absent from this report as it is redundant information.

To obtain the results of intensity of importance between indicators, three conditions were measured: (i) the value of the indicator located in the cell a_ij is equal to the value of the indicator of the cell a_ji; (ii) the value of a_ij is greater than a_ji; (iii) the value of a_ij is less than a_ji. the conditions observed in the paired comparison and the values allocated as result of the comparison are shown in

Table 2. Paired comparison of indicators and resulting values rules. Source: Adapted from Saaty and Vargas [128].

Rule		Result of the Comparison
$a_{ij}=a_{ji}$		$x_{ij}=1$
$a_{ij}>a_{ji}$	$a_{ij}-a_{ji}=1$	$x_{ij}=3$
	$a_{ij}-a_{ji}=2$	$x_{ij}=5$
	$a_{ij}-a_{ji}=3$	$x_{ij}=7$
	$a_{ij}-a_{ji}=4$	$x_{ij}=9$
$a_{ij}<a_{ji}$		$x_{ij}=\frac{1}{a_{ji}-a_{ij}}$

.

The sum of the 58 values of each indicator revealed the importance of each indicator for strong sustainability. The results of the paired comparison of indicators are shown in

Table A3. Results of the pairwise comparison of EXPERTs’ judgments.

	INDICATORS
EXPERTS	E01	E02	E03	E04	E05	E06	E07	A01	A02	A03	A04	A05	A06	A07	A08	A09	A10	A11	A12	A13	A14	S01	S02	S03	S04	S05	S06	S07	S08	S09	S10	S11
EXPERT 1	21.9	21.9	21.9	21.9	52.3	21.9	21.9	99.0	99.0	52.3	99.0	52.3	52.3	99.0	99.0	21.9	99.0	99.0	99.0	52.3	99.0	4.8	21.9	52.3	21.9	99.0	52.3	99.0	21.9	99.0	9.6	99.0
EXPERT 2	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	6.2	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0	35.0
EXPERT 3	9.2	9.2	9.2	9.2	45.7	45.7	88.3	139.0	139.0	139.0	45.7	139.0	139.0	139.0	139.0	139.0	88.3	139.0	139.0	45.7	45.7	9.2	45.7	45.7	45.7	45.7	45.7	45.7	45.7	45.7	9.2	45.7
EXPERT 4	10.4	10.4	10.4	10.4	10.4	10.4	10.4	159.0	159.0	159.0	53.4	53.4	53.4	53.4	159.0	159.0	159.0	53.4	53.4	53.4	159.0	53.4	53.4	53.4	53.4	53.4	53.4	53.4	53.4	53.4	53.4	53.4
EXPERT 5	19.8	19.8	19.8	19.8	19.8	19.8	7.6	84.3	84.3	143.0	84.3	84.3	143.0	84.3	45.1	143.0	143.0	84.3	84.3	45.1	84.3	7.6	45.1	84.3	19.8	84.3	7.6	45.1	84.3	84.3	45.1	84.3
EXPERT 6	15.1	15.1	15.1	15.1	15.1	15.1	43.7	89.0	89.0	89.0	89.0	89.0	89.0	89.0	89.0	89.0	89.0	89.0	89.0	43.7	89.0	15.1	15.1	15.1	43.7	43.7	43.7	15.1	43.7	89.0	15.1	43.7
EXPERT 7	10.2	52.5	10.2	93.0	10.2	52.5	10.2	145.0	52.5	145.0	93.0	145.0	145.0	52.5	52.5	27.3	145.0	52.5	52.5	145.0	52.5	10.2	145.0	10.2	52.5	145.0	27.3	93.0	93.0	93.0	10.2	145.0
EXPERT 8	29.7	29.7	29.7	29.7	68.3	11.8	29.7	29.7	119.0	119.0	29.7	68.3	119.0	119.0	119.0	119.0	119.0	68.3	119.0	119.0	119.0	5.4	11.8	29.7	29.7	29.7	68.3	29.7	11.8	29.7	68.3	29.7
EXPERT 9	44.9	44.9	44.9	44.9	44.9	44.9	44.9	147.0	147.0	91.0	147.0	91.0	44.9	44.9	44.9	147.0	44.9	91.0	91.0	44.9	44.9	8.2	91.0	20.5	8.2	8.2	147.0	8.2	44.9	20.5	147.0	20.5
EXPERT 10	10.6	10.6	10.6	10.6	29.3	10.6	57.0	97.0	97.0	97.0	97.0	57.0	97.0	97.0	97.0	97.0	97.0	97.0	97.0	29.3	97.0	10.6	29.3	97.0	10.6	97.0	97.0	97.0	97.0	97.0	29.3	29.3
EXPERT 11	54.3	101.0	22.6	101.0	22.6	9.7	54.3	101.0	101.0	22.6	22.6	101.0	101.0	101.0	101.0	101.0	101.0	54.3	22.6	101.0	22.6	4.8	22.6	54.3	54.3	101.0	22.6	22.6	22.6	22.6	54.3	101.0
EXPERT 12	27.7	27.7	9.5	27.7	27.7	77.0	77.0	27.7	27.7	77.0	77.0	27.7	27.7	27.7	77.0	27.7	77.0	27.7	27.7	27.7	77.0	9.5	27.7	77.0	27.7	77.0	77.0	77.0	27.7	27.7	27.7	27.7
EXPERT 13	15.9	38.3	85.0	6.8	85.0	6.8	38.3	38.3	15.9	38.3	85.0	85.0	38.3	85.0	38.3	38.3	38.3	38.3	85.0	85.0	38.3	85.0	38.3	38.3	15.9	85.0	85.0	85.0	15.9	85.0	38.3	85.0
EXPERT 14	15.9	38.3	85.0	6.8	85.0	6.8	38.3	38.3	15.9	38.3	85.0	85.0	38.3	85.0	38.3	38.3	38.3	38.3	85.0	85.0	38.3	85.0	38.3	38.3	15.9	85.0	85.0	85.0	15.9	85.0	38.3	85.0
EXPERT 15	9.6	26.3	63.7	63.7	26.3	9.6	26.3	26.3	63.7	121.0	26.3	63.7	63.7	26.3	26.3	26.3	26.3	26.3	63.7	121.0	121.0	63.7	63.7	121.0	63.7	63.7	63.7	9.6	9.6	63.7	121.0	63.7
EXPERT 16	56.1	56.1	143.0	143.0	11.4	11.4	56.1	143.0	143.0	56.1	143.0	143.0	143.0	143.0	56.1	95.0	95.0	143.0	143.0	56.1	56.1	11.4	56.1	56.1	143.0	143.0	11.4	11.4	11.4	11.4	11.4	11.4
EXPERT 17	16.7	16.7	16.7	16.7	16.7	16.7	16.7	187.0	84.1	84.1	187.0	187.0	187.0	187.0	84.1	132.3	132.3	187.0	187.0	84.1	84.1	16.7	84.1	84.1	16.7	84.1	16.7	16.7	16.7	16.7	16.7	16.7
EXPERT 18	14.8	14.8	33.7	107.0	6.0	6.0	63.0	107.0	107.0	63.0	107.0	107.0	107.0	107.0	107.0	107.0	107.0	107.0	107.0	33.7	33.7	14.8	107.0	14.8	107.0	107.0	33.7	33.7	63.0	63.0	14.8	63.0
EXPERT 19	53.0	53.0	53.0	53.0	21.0	8.5	8.5	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	21.0	21.0	53.0	21.0	53.0	53.0	53.0	53.0	21.0	8.5	53.0
EXPERT 20	17.7	17.7	76.3	7.1	17.7	7.1	38.7	131.0	131.0	76.3	76.3	38.7	76.3	131.0	76.3	76.3	131.0	38.7	17.7	76.3	38.7	7.1	38.7	76.3	38.7	131.0	38.7	131.0	76.3	38.7	76.3	131.0
EXPERT 21	9.8	9.8	9.8	9.8	9.8	9.8	51.8	167.0	51.8	51.8	51.8	51.8	51.8	51.8	167.0	167.0	167.0	51.8	51.8	51.8	51.8	51.8	51.8	51.8	51.8	51.8	51.8	51.8	51.8	51.8	51.8	51.8
EXPERT 22	53.0	53.0	53.0	53.0	21.0	8.5	8.5	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	21.0	21.0	53.0	21.0	53.0	53.0	53.0	53.0	21.0	8.5	53.0
EXPERT 23	43.0	16.1	16.1	16.1	43.0	43.0	91.0	91.0	43.0	91.0	43.0	91.0	91.0	91.0	91.0	43.0	91.0	91.0	91.0	16.1	6.6	43.0	43.0	91.0	16.1	43.0	91.0	43.0	16.1	43.0	16.1	16.1
EXPERT 24	26.2	26.2	99.0	99.0	9.6	9.6	26.2	99.0	99.0	26.2	99.0	99.0	99.0	99.0	99.0	99.0	99.0	99.0	99.0	26.2	26.2	9.6	99.0	9.6	99.0	99.0	26.2	26.2	56.3	56.3	9.6	56.3
EXPERT 25	87.0	40.6	40.6	7.4	87.0	7.4	87.0	147.0	40.6	147.0	40.6	87.0	87.0	17.9	87.0	147.0	40.6	87.0	87.0	40.6	40.6	7.4	17.9	40.6	40.6	40.6	40.6	17.9	87.0	40.6	40.6	40.6
EXPERT 26	29.7	29.7	29.7	29.7	68.3	11.8	29.7	29.7	119.0	119.0	29.7	68.3	119.0	119.0	119.0	119.0	119.0	68.3	119.0	119.0	119.0	5.4	11.8	29.7	29.7	29.7	68.3	29.7	11.8	29.7	68.3	29.7
EXPERT 27	8.6	8.6	22.7	22.7	56.3	8.6	56.3	111.0	56.3	56.3	56.3	22.7	56.3	111.0	111.0	111.0	22.7	56.3	22.7	56.3	56.3	22.7	56.3	56.3	22.7	111.0	22.7	56.3	56.3	111.0	22.7	111.0
EXPERT 28	31.4	66.3	66.3	66.3	121.0	12.7	12.7	121.0	31.4	31.4	66.3	66.3	66.3	12.7	5.6	66.3	121.0	31.4	31.4	12.7	121.0	31.4	66.3	121.0	31.4	66.3	66.3	121.0	66.3	121.0	66.3	12.7
EXPERT 29	23.0	59.0	23.0	59.0	59.0	23.0	59.0	59.0	59.0	59.0	23.0	59.0	59.0	59.0	59.0	8.9	59.0	59.0	59.0	8.9	59.0	8.9	59.0	23.0	23.0	59.0	23.0	59.0	59.0	59.0	59.0	23.0
EXPERT 30	13.6	29.7	29.7	60.3	13.6	5.9	5.9	60.3	60.3	111.0	60.3	111.0	111.0	111.0	60.3	111.0	60.3	111.0	60.3	29.7	60.3	13.6	111.0	29.7	29.7	111.0	60.3	111.0	29.7	111.0	60.3	60.3
EXPERT 31	15.9	38.3	85.0	6.8	85.0	6.8	38.3	38.3	15.9	38.3	85.0	85.0	38.3	85.0	38.3	38.3	38.3	38.3	85.0	85.0	38.3	85.0	38.3	38.3	15.9	85.0	85.0	85.0	15.9	85.0	38.3	85.0
EXPERT 32	143.0	143.0	143.0	143.0	95.0	143.0	143.0	143.0	143.0	143.0	143.0	95.0	143.0	143.0	95.0	11.0	11.0	30.8	58.7	11.0	11.0	58.7	58.7	58.7	11.0	95.0	30.8	30.8	11.0	11.0	30.8	11.0
EXPERT 33	16.5	16.5	16.5	16.5	45.0	6.8	16.5	97.0	97.0	97.0	97.0	45.0	45.0	97.0	45.0	97.0	97.0	45.0	45.0	97.0	45.0	16.5	45.0	45.0	16.5	45.0	45.0	45.0	16.5	45.0	45.0	97.0
EXPERT 34	99.0	51.0	51.0	51.0	51.0	21.4	21.4	99.0	99.0	99.0	51.0	99.0	51.0	51.0	21.4	51.0	21.4	21.4	99.0	99.0	99.0	4.8	9.5	21.4	21.4	99.0	51.0	21.4	21.4	99.0	99.0	99.0
EXPERT 35	53.0	53.0	53.0	53.0	21.0	8.5	8.5	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	53.0	21.0	21.0	53.0	21.0	53.0	53.0	53.0	53.0	21.0	8.5	53.0
EXPERT 36	25.6	9.4	9.4	25.6	25.6	98.3	55.1	157.0	157.0	98.3	98.3	98.3	55.1	55.1	55.1	157.0	98.3	98.3	98.3	157.0	98.3	25.6	25.6	55.1	9.4	25.6	98.3	9.4	9.4	25.6	55.1	98.3
EXPERT 37	74.3	74.3	35.4	35.4	74.3	6.9	35.4	74.3	35.4	129.0	129.0	74.3	74.3	35.4	35.4	129.0	74.3	129.0	35.4	129.0	6.9	6.9	129.0	35.4	35.4	35.4	129.0	35.4	16.5	16.5	74.3	74.3
EXPERT 38	49.3	49.3	49.3	49.3	49.3	7.5	49.3	97.0	97.0	157.0	97.0	97.0	97.0	157.0	157.0	49.3	97.0	97.0	97.0	49.3	49.3	19.4	19.4	19.4	19.4	19.4	19.4	19.4	19.4	19.4	7.5	49.3
EXPERT 39	47.7	97.0	97.0	97.0	17.5	17.5	47.7	47.7	47.7	97.0	17.5	47.7	47.7	17.5	47.7	97.0	97.0	17.5	17.5	97.0	17.5	47.7	17.5	47.7	47.7	97.0	97.0	17.5	6.9	17.5	97.0	97.0
EXPERT 40	15.9	38.3	85.0	6.8	85.0	6.8	38.3	38.3	15.9	38.3	85.0	85.0	38.3	85.0	38.3	38.3	38.3	38.3	85.0	85.0	38.3	85.0	38.3	38.3	15.9	85.0	85.0	85.0	15.9	85.0	38.3	85.0
EXPERT 41	18.3	18.3	81.0	81.0	18.3	18.3	18.3	81.0	81.0	18.3	81.0	81.0	81.0	81.0	81.0	81.0	81.0	81.0	81.0	81.0	81.0	4.4	81.0	81.0	18.3	81.0	43.7	18.3	18.3	81.0	18.3	81.0
EXPERT 42	99.0	51.0	51.0	51.0	51.0	21.4	21.4	99.0	99.0	99.0	51.0	99.0	51.0	51.0	21.4	51.0	21.4	21.4	99.0	99.0	99.0	4.8	9.5	21.4	21.4	99.0	51.0	21.4	21.4	99.0	99.0	99.0
EXPERT 43	29.7	29.7	29.7	29.7	68.3	11.8	29.7	29.7	119.0	119.0	29.7	68.3	119.0	119.0	119.0	119.0	119.0	68.3	119.0	119.0	119.0	5.4	11.8	29.7	29.7	29.7	68.3	29.7	11.8	29.7	68.3	29.7
EXPERT 44	11.3	83.0	43.0	43.0	83.0	5.1	43.0	83.0	83.0	83.0	83.0	21.9	43.0	83.0	43.0	21.9	83.0	83.0	21.9	83.0	43.0	5.1	43.0	83.0	83.0	83.0	83.0	83.0	21.9	83.0	83.0	83.0
EXPERT 45	11.0	30.9	30.9	11.0	30.9	63.5	63.5	163.0	163.0	107.0	163.0	63.5	107.0	163.0	163.0	107.0	163.0	63.5	107.0	30.9	30.9	11.0	11.0	11.0	11.0	107.0	63.5	107.0	30.9	107.0	30.9	11.0
EXPERT 46	10.0	10.0	10.0	10.0	10.0	10.0	51.8	131.0	131.0	131.0	131.0	131.0	131.0	131.0	131.0	131.0	131.0	131.0	131.0	51.8	51.8	27.3	131.0	27.3	10.0	131.0	51.8	51.8	85.7	85.7	27.3	85.7
EXPERT 47	29.7	29.7	29.7	29.7	68.3	11.8	29.7	29.7	119.0	119.0	29.7	68.3	119.0	119.0	119.0	119.0	119.0	68.3	119.0	119.0	119.0	5.4	11.8	29.7	29.7	29.7	68.3	29.7	11.8	29.7	68.3	29.7
EXPERT 48	67.0	23.0	67.0	23.0	23.0	23.0	67.0	67.0	67.0	23.0	23.0	67.0	67.0	67.0	67.0	23.0	67.0	23.0	67.0	8.3	67.0	67.0	23.0	67.0	23.0	23.0	23.0	67.0	23.0	23.0	23.0	23.0
EXPERT 49	19.1	19.1	19.1	19.1	19.1	19.1	19.1	85.0	85.0	85.0	85.0	85.0	85.0	85.0	85.0	85.0	85.0	85.0	85.0	85.0	85.0	4.5	85.0	85.0	19.1	85.0	46.3	19.1	19.1	85.0	19.1	85.0
EXPERT 50	20.3	20.3	7.8	20.3	20.3	7.8	52.3	20.3	20.3	103.0	52.3	52.3	52.3	103.0	20.3	103.0	103.0	20.3	20.3	103.0	52.3	20.3	52.3	103.0	52.3	103.0	52.3	52.3	103.0	52.3	103.0	103.0
EXPERT 51	13.7	13.7	13.7	13.7	40.0	13.7	40.0	187.0	187.0	79.3	127.0	127.0	127.0	127.0	127.0	187.0	127.0	127.0	127.0	79.3	79.3	13.7	40.0	13.7	13.7	13.7	40.0	13.7	40.0	40.0	40.0	40.0
EXPERT 52	87.0	40.6	40.6	7.4	87.0	7.4	87.0	147.0	40.6	147.0	40.6	87.0	87.0	17.9	87.0	147.0	40.6	87.0	87.0	40.6	40.6	7.4	17.9	40.6	40.6	40.6	40.6	17.9	87.0	40.6	40.6	40.6
EXPERT 53	9.5	61.7	9.5	61.7	25.7	9.5	61.7	119.0	61.7	119.0	61.7	119.0	119.0	119.0	61.7	61.7	61.7	61.7	61.7	25.7	25.7	9.5	25.7	61.7	25.7	25.7	25.7	25.7	25.7	61.7	61.7	61.7
EXPERT 54	19.1	19.1	19.1	19.1	19.1	19.1	19.1	85.0	85.0	85.0	85.0	85.0	85.0	85.0	85.0	85.0	85.0	85.0	85.0	85.0	85.0	4.5	85.0	85.0	19.1	85.0	46.3	19.1	19.1	85.0	19.1	85.0
EXPERT 55	74.3	74.3	35.4	35.4	74.3	6.9	35.4	74.3	35.4	129.0	129.0	74.3	74.3	35.4	35.4	129.0	74.3	129.0	35.4	129.0	6.9	6.9	129.0	35.4	35.4	35.4	129.0	35.4	16.5	16.5	74.3	74.3
EXPERT 56	87.0	40.6	40.6	7.4	87.0	7.4	87.0	147.0	40.6	147.0	40.6	87.0	87.0	17.9	87.0	147.0	40.6	87.0	87.0	40.6	40.6	7.4	17.9	40.6	40.6	40.6	40.6	17.9	87.0	40.6	40.6	40.6
EXPERT 57	19.8	19.8	19.8	19.8	19.8	19.8	7.6	84.3	84.3	143.0	84.3	84.3	143.0	84.3	45.1	143.0	143.0	84.3	84.3	45.1	84.3	7.6	45.1	84.3	19.8	84.3	7.6	45.1	84.3	84.3	45.1	84.3
EXPERT 58	29.7	29.7	29.7	29.7	68.3	11.8	29.7	29.7	119.0	119.0	29.7	68.3	119.0	119.0	119.0	119.0	119.0	68.3	119.0	119.0	119.0	5.4	11.8	29.7	29.7	29.7	68.3	29.7	11.8	29.7	68.3	29.7

.

The next step was to establish the weight of each dimension, which was obtained through the arithmetic mean of the values of its indicators. This action aimed to eliminate the impact caused in the calculation due to the difference in the number of indicators in each dimension: seven in economic, fourteen in environmental and eleven in social dimensions. The procedure for aggregating indicators in each dimension was observed in the study by Jiang et al. [106], which measured the individual performance of sustainability dimensions in the main component analysis.

Thus, the average values of the economic, environmental and social dimensions in strong sustainability were calculated, as indicated in Equations (3)–(5).

$${\overline{w}}_E= \left(\frac{{{\displaystyle \sum }}_{i=1}^7{X}_{E_i}}{7}\right)$$

(3)

$${\overline{w}}_N= \left(\frac{{{\displaystyle \sum }}_{i=1}^{14}{X}_{N_i}}{14}\right)$$

(4)

$${\overline{w}}_S= \left(\frac{{{\displaystyle \sum }}_{i=1}^{11}{X}_{S_i}}{11}\right)$$

(5)

In which:

• $X_E{}_i \mathrm{is} \mathrm{the} \mathrm{sum} \mathrm{of} \mathrm{the} 58 \mathrm{comparisons} \mathrm{of} \mathrm{the} \mathrm{economic} \mathrm{indicator} E_i$
• $X_N{}_i \mathrm{is} \mathrm{the} \mathrm{sum} \mathrm{of} \mathrm{the} 58 \mathrm{comparisons} \mathrm{of} \mathrm{the} \mathrm{environmental} \mathrm{indicator} N_i$
• $X_S{}_i \mathrm{is} \mathrm{the} \mathrm{sum} \mathrm{of} \mathrm{the} 58 \mathrm{comparisons} \mathrm{of} \mathrm{the} \mathrm{social} \mathrm{indicator} S_i$
• ${\overline{w}}_E \mathrm{is} \mathrm{the} \mathrm{weight} \mathrm{of} \mathrm{the} \mathrm{economic} \mathrm{dimension}$
• ${\overline{w}}_N \mathrm{is} \mathrm{the} \mathrm{weight} \mathrm{of} \mathrm{the} \mathrm{environmental} \mathrm{dimension}$
• ${\overline{w}}_S \mathrm{the} \mathrm{weight} \mathrm{of} \mathrm{the} \mathrm{social} \mathrm{dimension}$

The obtained results were converted into percentage values, to reveal the participation of each dimension in strong sustainability. The models of sustainability evaluation in manufacturing referenced in this research considered weights equivalent to the dimensions of sustainability. Due to the absence of strong sustainability evaluation models, the calculation of dimension participation was suggested by this research. Thus, it was possible to verify whether the dimensions are equivalent or different in terms of relevance to strong sustainability.

The percentage participation of the economic dimension was calculated by dividing its average weight by the sum of the average weights of the three dimensions, as shown in Equation (6). The calculations of the environmental and social dimensions followed the same method.

$$\%ECO= \frac{{\overline{w}}_E}{{\overline{w}}_E+{\overline{w}}_N+{\overline{w}}_S} \times 100$$

(6)

The sum of the percentage participation of the three dimensions results in the strong sustainability total. The method used by Krajnc and Glavic [109] considered that the sustainability index is obtained by the arithmetic mean of the three dimensions, in percentage scale. The procedure presented in this research was adapted due to the values of the dimensions already being defined in percentages. Thus, the sum of the three dimensions is 100%, as shown in Equation (7).

$$Strong Sustainability =\%ECO+\%ENV+\%SOC$$

(7)

The weight of the indicators was obtained by dividing the value of the indicator by the sum of the indicator values of the same dimension, multiplied by the weight of the dimension. The weight calculation of the economic indicator E₁ was performed according to Equation (8). The weights of the other 31 indicators were obtained following the same procedure. The results are shown in

Table A4. Weights of indicators through the sum of pairwise comparison.

INDICATORS
Description	E01	E02	E03	E04	E05	E06	E07	N01	N02	N03	N04	N05	N06	N07	N08	N09	N10	N11	N12	N13	N14	S01	S02	S03	S04	S05	S06	S07	S08	S09	S10	S11
	0.012	0.012	0.012	0.012	0.028	0.012	0.012	0.052	0.052	0.028	0.052	0.028	0.028	0.052	0.052	0.012	0.052	0.052	0.052	0.028	0.052	0.003	0.012	0.028	0.012	0.052	0.028	0.052	0.012	0.052	0.005	0.052
	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.006	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032	0.032
	0.004	0.004	0.004	0.004	0.020	0.020	0.039	0.061	0.061	0.061	0.020	0.061	0.061	0.061	0.061	0.061	0.039	0.061	0.061	0.020	0.020	0.004	0.020	0.020	0.020	0.020	0.020	0.020	0.020	0.020	0.004	0.020
	0.005	0.005	0.005	0.005	0.005	0.005	0.005	0.074	0.074	0.074	0.025	0.025	0.025	0.025	0.074	0.074	0.074	0.025	0.025	0.025	0.074	0.025	0.025	0.025	0.025	0.025	0.025	0.025	0.025	0.025	0.025	0.025
	0.010	0.010	0.010	0.010	0.010	0.010	0.004	0.041	0.041	0.070	0.041	0.041	0.070	0.041	0.022	0.070	0.070	0.041	0.041	0.022	0.041	0.004	0.022	0.041	0.010	0.041	0.004	0.022	0.041	0.041	0.022	0.041
	0.009	0.009	0.009	0.009	0.009	0.009	0.025	0.052	0.052	0.052	0.052	0.052	0.052	0.052	0.052	0.052	0.052	0.052	0.052	0.025	0.052	0.009	0.009	0.009	0.025	0.025	0.025	0.009	0.025	0.052	0.009	0.025
	0.004	0.022	0.004	0.039	0.004	0.022	0.004	0.061	0.022	0.061	0.039	0.061	0.061	0.022	0.022	0.012	0.061	0.022	0.022	0.061	0.022	0.004	0.061	0.004	0.022	0.061	0.012	0.039	0.039	0.039	0.004	0.061
	0.015	0.015	0.015	0.015	0.035	0.006	0.015	0.015	0.061	0.061	0.015	0.035	0.061	0.061	0.061	0.061	0.061	0.035	0.061	0.061	0.061	0.003	0.006	0.015	0.015	0.015	0.035	0.015	0.006	0.015	0.035	0.015
	0.022	0.022	0.022	0.022	0.022	0.022	0.022	0.071	0.071	0.044	0.071	0.044	0.022	0.022	0.022	0.071	0.022	0.044	0.044	0.022	0.022	0.004	0.044	0.010	0.004	0.004	0.071	0.004	0.022	0.010	0.071	0.010
	0.005	0.005	0.005	0.005	0.014	0.005	0.027	0.047	0.047	0.047	0.047	0.027	0.047	0.047	0.047	0.047	0.047	0.047	0.047	0.014	0.047	0.005	0.014	0.047	0.005	0.047	0.047	0.047	0.047	0.047	0.014	0.014
	0.029	0.053	0.012	0.053	0.012	0.005	0.029	0.053	0.053	0.012	0.012	0.053	0.053	0.053	0.053	0.053	0.053	0.029	0.012	0.053	0.012	0.003	0.012	0.029	0.029	0.053	0.012	0.012	0.012	0.012	0.029	0.053
	0.020	0.020	0.007	0.020	0.020	0.055	0.055	0.020	0.020	0.055	0.055	0.020	0.020	0.020	0.055	0.020	0.055	0.020	0.020	0.020	0.055	0.007	0.020	0.055	0.020	0.055	0.055	0.055	0.020	0.020	0.020	0.020
	0.009	0.023	0.051	0.004	0.051	0.004	0.023	0.023	0.009	0.023	0.051	0.051	0.023	0.051	0.023	0.023	0.023	0.023	0.051	0.051	0.023	0.051	0.023	0.023	0.009	0.051	0.051	0.051	0.009	0.051	0.023	0.051
	0.009	0.023	0.051	0.004	0.051	0.004	0.023	0.023	0.009	0.023	0.051	0.051	0.023	0.051	0.023	0.023	0.023	0.023	0.051	0.051	0.023	0.051	0.023	0.023	0.009	0.051	0.051	0.051	0.009	0.051	0.023	0.051
	0.006	0.015	0.037	0.037	0.015	0.006	0.015	0.015	0.037	0.070	0.015	0.037	0.037	0.015	0.015	0.015	0.015	0.015	0.037	0.070	0.070	0.037	0.037	0.070	0.037	0.037	0.037	0.006	0.006	0.037	0.070	0.037
	0.022	0.022	0.057	0.057	0.005	0.005	0.022	0.057	0.057	0.022	0.057	0.057	0.057	0.057	0.022	0.038	0.038	0.057	0.057	0.022	0.022	0.005	0.022	0.022	0.057	0.057	0.005	0.005	0.005	0.005	0.005	0.005
	0.007	0.007	0.007	0.007	0.007	0.007	0.007	0.075	0.034	0.034	0.075	0.075	0.075	0.075	0.034	0.053	0.053	0.075	0.075	0.034	0.034	0.007	0.034	0.034	0.007	0.034	0.007	0.007	0.007	0.007	0.007	0.007
	0.007	0.007	0.015	0.049	0.003	0.003	0.029	0.049	0.049	0.029	0.049	0.049	0.049	0.049	0.049	0.049	0.049	0.049	0.049	0.015	0.015	0.007	0.049	0.007	0.049	0.049	0.015	0.015	0.029	0.029	0.007	0.029
	0.038	0.038	0.038	0.038	0.015	0.006	0.006	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.015	0.015	0.038	0.015	0.038	0.038	0.038	0.038	0.015	0.006	0.038
	0.008	0.008	0.037	0.003	0.008	0.003	0.019	0.063	0.063	0.037	0.037	0.019	0.037	0.063	0.037	0.037	0.063	0.019	0.008	0.037	0.019	0.003	0.019	0.037	0.019	0.063	0.019	0.063	0.037	0.019	0.037	0.063
	0.005	0.005	0.005	0.005	0.005	0.005	0.028	0.089	0.028	0.028	0.028	0.028	0.028	0.028	0.089	0.089	0.089	0.028	0.028	0.028	0.028	0.028	0.028	0.028	0.028	0.028	0.028	0.028	0.028	0.028	0.028	0.028
	0.038	0.038	0.038	0.038	0.015	0.006	0.006	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.015	0.015	0.038	0.015	0.038	0.038	0.038	0.038	0.015	0.006	0.038
	0.025	0.009	0.009	0.009	0.025	0.025	0.054	0.054	0.025	0.054	0.025	0.054	0.054	0.054	0.054	0.025	0.054	0.054	0.054	0.009	0.004	0.025	0.025	0.054	0.009	0.025	0.054	0.025	0.009	0.025	0.009	0.009
	0.013	0.013	0.049	0.049	0.005	0.005	0.013	0.049	0.049	0.013	0.049	0.049	0.049	0.049	0.049	0.049	0.049	0.049	0.049	0.013	0.013	0.005	0.049	0.005	0.049	0.049	0.013	0.013	0.028	0.028	0.005	0.028
	0.047	0.022	0.022	0.004	0.047	0.004	0.047	0.079	0.022	0.079	0.022	0.047	0.047	0.010	0.047	0.079	0.022	0.047	0.047	0.022	0.022	0.004	0.010	0.022	0.022	0.022	0.022	0.010	0.047	0.022	0.022	0.022
	0.015	0.015	0.015	0.015	0.035	0.006	0.015	0.015	0.061	0.061	0.015	0.035	0.061	0.061	0.061	0.061	0.061	0.035	0.061	0.061	0.061	0.003	0.006	0.015	0.015	0.015	0.035	0.015	0.006	0.015	0.035	0.015
	0.005	0.005	0.013	0.013	0.032	0.005	0.032	0.064	0.032	0.032	0.032	0.013	0.032	0.064	0.064	0.064	0.013	0.032	0.013	0.032	0.032	0.013	0.032	0.032	0.013	0.064	0.013	0.032	0.032	0.064	0.013	0.064
	0.016	0.034	0.034	0.034	0.063	0.007	0.007	0.063	0.016	0.016	0.034	0.034	0.034	0.007	0.003	0.034	0.063	0.016	0.016	0.007	0.063	0.016	0.034	0.063	0.016	0.034	0.034	0.063	0.034	0.063	0.034	0.007
	0.016	0.041	0.016	0.041	0.041	0.016	0.041	0.041	0.041	0.041	0.016	0.041	0.041	0.041	0.041	0.006	0.041	0.041	0.041	0.006	0.041	0.006	0.041	0.016	0.016	0.041	0.016	0.041	0.041	0.041	0.041	0.016
	0.007	0.015	0.015	0.030	0.007	0.003	0.003	0.030	0.030	0.055	0.030	0.055	0.055	0.055	0.030	0.055	0.030	0.055	0.030	0.015	0.030	0.007	0.055	0.015	0.015	0.055	0.030	0.055	0.015	0.055	0.030	0.030
	0.009	0.023	0.051	0.004	0.051	0.004	0.023	0.023	0.009	0.023	0.051	0.051	0.023	0.051	0.023	0.023	0.023	0.023	0.051	0.051	0.023	0.051	0.023	0.023	0.009	0.051	0.051	0.051	0.009	0.051	0.023	0.051
	0.056	0.056	0.056	0.056	0.037	0.056	0.056	0.056	0.056	0.056	0.056	0.037	0.056	0.056	0.037	0.004	0.004	0.012	0.023	0.004	0.004	0.023	0.023	0.023	0.004	0.037	0.012	0.012	0.004	0.004	0.012	0.004
	0.010	0.010	0.010	0.010	0.027	0.004	0.010	0.059	0.059	0.059	0.059	0.027	0.027	0.059	0.027	0.059	0.059	0.027	0.027	0.059	0.027	0.010	0.027	0.027	0.010	0.027	0.027	0.027	0.010	0.027	0.027	0.059
	0.053	0.028	0.028	0.028	0.028	0.012	0.012	0.053	0.053	0.053	0.028	0.053	0.028	0.028	0.012	0.028	0.012	0.012	0.053	0.053	0.053	0.003	0.005	0.012	0.012	0.053	0.028	0.012	0.012	0.053	0.053	0.053
	0.038	0.038	0.038	0.038	0.015	0.006	0.006	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.038	0.015	0.015	0.038	0.015	0.038	0.038	0.038	0.038	0.015	0.006	0.038
	0.012	0.004	0.004	0.012	0.012	0.045	0.025	0.072	0.072	0.045	0.045	0.045	0.025	0.025	0.025	0.072	0.045	0.045	0.045	0.072	0.045	0.012	0.012	0.025	0.004	0.012	0.045	0.004	0.004	0.012	0.025	0.045
	0.037	0.037	0.018	0.018	0.037	0.003	0.018	0.037	0.018	0.064	0.064	0.037	0.037	0.018	0.018	0.064	0.037	0.064	0.018	0.064	0.003	0.003	0.064	0.018	0.018	0.018	0.064	0.018	0.008	0.008	0.037	0.037
	0.026	0.026	0.026	0.026	0.026	0.004	0.026	0.050	0.050	0.081	0.050	0.050	0.050	0.081	0.081	0.026	0.050	0.050	0.050	0.026	0.026	0.010	0.010	0.010	0.010	0.010	0.010	0.010	0.010	0.010	0.004	0.026
	0.028	0.056	0.056	0.056	0.010	0.010	0.028	0.028	0.028	0.056	0.010	0.028	0.028	0.010	0.028	0.056	0.056	0.010	0.010	0.056	0.010	0.028	0.010	0.028	0.028	0.056	0.056	0.010	0.004	0.010	0.056	0.056
	0.009	0.023	0.051	0.004	0.051	0.004	0.023	0.023	0.009	0.023	0.051	0.051	0.023	0.051	0.023	0.023	0.023	0.023	0.051	0.051	0.023	0.051	0.023	0.023	0.009	0.051	0.051	0.051	0.009	0.051	0.023	0.051
	0.010	0.010	0.044	0.044	0.010	0.010	0.010	0.044	0.044	0.010	0.044	0.044	0.044	0.044	0.044	0.044	0.044	0.044	0.044	0.044	0.044	0.002	0.044	0.044	0.010	0.044	0.024	0.010	0.010	0.044	0.010	0.044
	0.053	0.028	0.028	0.028	0.028	0.012	0.012	0.053	0.053	0.053	0.028	0.053	0.028	0.028	0.012	0.028	0.012	0.012	0.053	0.053	0.053	0.003	0.005	0.012	0.012	0.053	0.028	0.012	0.012	0.053	0.053	0.053
	0.015	0.015	0.015	0.015	0.035	0.006	0.015	0.015	0.061	0.061	0.015	0.035	0.061	0.061	0.061	0.061	0.061	0.035	0.061	0.061	0.061	0.003	0.006	0.015	0.015	0.015	0.035	0.015	0.006	0.015	0.035	0.015
	0.006	0.044	0.023	0.023	0.044	0.003	0.023	0.044	0.044	0.044	0.044	0.012	0.023	0.044	0.023	0.012	0.044	0.044	0.012	0.044	0.023	0.003	0.023	0.044	0.044	0.044	0.044	0.044	0.012	0.044	0.044	0.044
	0.005	0.013	0.013	0.005	0.013	0.027	0.027	0.070	0.070	0.046	0.070	0.027	0.046	0.070	0.070	0.046	0.070	0.027	0.046	0.013	0.013	0.005	0.005	0.005	0.005	0.046	0.027	0.046	0.013	0.046	0.013	0.005
	0.004	0.004	0.004	0.004	0.004	0.004	0.021	0.052	0.052	0.052	0.052	0.052	0.052	0.052	0.052	0.052	0.052	0.052	0.052	0.021	0.021	0.011	0.052	0.011	0.004	0.052	0.021	0.021	0.034	0.034	0.011	0.034
	0.015	0.015	0.015	0.015	0.035	0.006	0.015	0.015	0.061	0.061	0.015	0.035	0.061	0.061	0.061	0.061	0.061	0.035	0.061	0.061	0.061	0.003	0.006	0.015	0.015	0.015	0.035	0.015	0.006	0.015	0.035	0.015
	0.049	0.017	0.049	0.017	0.017	0.017	0.049	0.049	0.049	0.017	0.017	0.049	0.049	0.049	0.049	0.017	0.049	0.017	0.049	0.006	0.049	0.049	0.017	0.049	0.017	0.017	0.017	0.049	0.017	0.017	0.017	0.017
	0.010	0.010	0.010	0.010	0.010	0.010	0.010	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.002	0.045	0.045	0.010	0.045	0.025	0.010	0.010	0.045	0.010	0.045
	0.011	0.011	0.004	0.011	0.011	0.004	0.030	0.011	0.011	0.058	0.030	0.030	0.030	0.058	0.011	0.058	0.058	0.011	0.011	0.058	0.030	0.011	0.030	0.058	0.030	0.058	0.030	0.030	0.058	0.030	0.058	0.058
	0.006	0.006	0.006	0.006	0.018	0.006	0.018	0.082	0.082	0.035	0.056	0.056	0.056	0.056	0.056	0.082	0.056	0.056	0.056	0.035	0.035	0.006	0.018	0.006	0.006	0.006	0.018	0.006	0.018	0.018	0.018	0.018
	0.047	0.022	0.022	0.004	0.047	0.004	0.047	0.079	0.022	0.079	0.022	0.047	0.047	0.010	0.047	0.079	0.022	0.047	0.047	0.022	0.022	0.004	0.010	0.022	0.022	0.022	0.022	0.010	0.047	0.022	0.022	0.022
	0.006	0.036	0.006	0.036	0.015	0.006	0.036	0.069	0.036	0.069	0.036	0.069	0.069	0.069	0.036	0.036	0.036	0.036	0.036	0.015	0.015	0.006	0.015	0.036	0.015	0.015	0.015	0.015	0.015	0.036	0.036	0.036
	0.010	0.010	0.010	0.010	0.010	0.010	0.010	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.002	0.045	0.045	0.010	0.045	0.025	0.010	0.010	0.045	0.010	0.045
	0.037	0.037	0.018	0.018	0.037	0.003	0.018	0.037	0.018	0.064	0.064	0.037	0.037	0.018	0.018	0.064	0.037	0.064	0.018	0.064	0.003	0.003	0.064	0.018	0.018	0.018	0.064	0.018	0.008	0.008	0.037	0.037
	0.047	0.022	0.022	0.004	0.047	0.004	0.047	0.079	0.022	0.079	0.022	0.047	0.047	0.010	0.047	0.079	0.022	0.047	0.047	0.022	0.022	0.004	0.010	0.022	0.022	0.022	0.022	0.010	0.047	0.022	0.022	0.022
	0.010	0.010	0.010	0.010	0.010	0.010	0.004	0.041	0.041	0.070	0.041	0.041	0.070	0.041	0.022	0.070	0.070	0.041	0.041	0.022	0.041	0.004	0.022	0.041	0.010	0.041	0.004	0.022	0.041	0.041	0.022	0.041
	0.015	0.015	0.015	0.015	0.035	0.006	0.015	0.015	0.061	0.061	0.015	0.035	0.061	0.061	0.061	0.061	0.061	0.035	0.061	0.061	0.061	0.003	0.006	0.015	0.015	0.015	0.035	0.015	0.006	0.015	0.035	0.015
Soma	1.093	1.172	1.293	1.187	1.355	0.618	1.286	2.729	2.434	2.734	2.234	2.423	2.509	2.527	2.338	2.670	2.568	2.149	2.359	2.095	1.960	0.699	1.427	1.581	1.016	2.106	1.737	1.474	1.195	1.723	1.422	1.889
Average of indicator	1.143							2.409														1.479
Share of dimensions	23%							48%														29%
Weight of indicators	0.031	0.033	0.037	0.034	0.038	0.018	0.037	0.039	0.035	0.039	0.032	0.034	0.036	0.036	0.033	0.038	0.036	0.031	0.033	0.030	0.028	0.013	0.026	0.029	0.018	0.038	0.031	0.027	0.022	0.031	0.026	0.034

.

$$w_{E_1}=\frac{X_E{}_1}{{{\displaystyle \sum }}_i^7{X}_E{}_i} \times \%ECO$$

(8)

In which:

• $w_{E_1}\mathrm{is} \mathrm{the} \mathrm{weight} \mathrm{of} \mathrm{the} \mathrm{indicator} E_1$
• $X_E{}_i \mathrm{is} \mathrm{the} \mathrm{sum} \mathrm{of} \mathrm{the} 58 \mathrm{comparisons} \mathrm{of} \mathrm{the} \mathrm{economic} \mathrm{indicator} E_i$
• $\%ECO \mathrm{is} \mathrm{the} \mathrm{participation} \mathrm{of} \mathrm{the} \mathrm{economic} \mathrm{dimension} \mathrm{in} \mathrm{strong} \mathrm{sustainability}$

In the economic dimension, profit (E05) was the indicator with the highest weight (0.038). The result of the expert analysis reflected the reason for the existence of the business. The profitability of the company is what ensures financial prosperity for maintaining the operation. In this dimension, market share (E06) was the least relevant indicator (0.018). This finding suggests that the level of competition in the segment has little influence on sustainability performance in manufacturing.

In the environmental dimension, the consumption of non-renewable materials (N01) was the indicator with the highest weight (0.039). This result emphasized the main concern of strong sustainability, which is the substitution of sources of non-renewable resources with renewables. Oppositely, the evaluation of suppliers (N14) was the indicator with the least relevance (0.028) in the environmental dimension. This result suggested that sustainability performance lies primarily in the internal results of the operation.

Occupational safety and health were emphasized in the social dimension. The indicator of number of accidents (S05) received the highest weight (0.038). This finding emphasized the concern with the integrity of employees. Thus, actions aimed at accident prevention are the ones that have the greatest positive impact on the best social outcome of the company. At the other extreme is the indicator of number of employees (S01), which was the least relevant (0.013) in the evaluation of the experts. This result showed that the size of the company has little influence on the sustainability of the business.

The measurement of the dimensions of strong sustainability allowed the quantitative comparison with weak sustainability. The result of the paired comparison of the notes attributed by the experts indicated that the environmental dimension is the most important for strong sustainability, with 48% participation, followed by the social (29%) and economic (23%) dimensions. By definition, weak sustainability implies the equivalence of the three dimensions and the substitutability or compensation between dimensions [17]. As a percentage scale, each dimension of weak sustainability corresponds to 33% of the total. The participation of the dimensions in weak and strong sustainability is shown in Figure 4.

4.4. Sustainability Index

The value of the Sustainability Index (SI) was obtained by means of the sum of each indicator multiplied by its weight, as shown in Equation (9). This calculation was referenced in the method used by Beekaroo et al. [110], which obtained the sustainability index through the weighted sum of economic, environmental and social indicators.

$$SI={\displaystyle \sum }_{i=1}^7{w}_{E_i}\times E_i+{\displaystyle \sum }_{i=1}^{14}{w}_{N_i}\times N_i+{\displaystyle \sum }_{i=1}^{11}{w}_{S_i}\times S_i$$

(9)

In which:

• w is the weight of indicator
• E_i is the economic indicator (i = 1, …, 7)
• N_i is the environmental indicator (i = 1, …, 14)
• i is the index identifying the single indicator
• S_i is the social indicator (i = 1, …,11)

The equation that calculates the Sustainability Index (SI) limits the influence of the economic factor to 23%, environmental factor to 48% and social factor to 29% in the total sustainability result. The difference in the number of indicators used to measure the performance of the economic, environmental and social dimensions does not influence the sustainability result.

5. Multi-Case Study

The Brazilian companies on this study are named as “BR1” and “BR2”, the Italian companies as “IT1” and “IT2”. The four companies are multinational companies in the automotive sector present in five continents. Its products equip light, heavy and off-road vehicles. Additionally, the four companies are ISO14001 certified and issue a sustainability report.

5.1. Economic, Environmental and Social Results

The analysis of economic data from BR1 indicated sales decline (E01) in 2015 and 2016, at what time sales volume began to increase in consecutive years. Revenue (E02) reflected the trend presented in the sales indicator. Operating costs (E03) fluctuated, with the worst result in 2016, influenced by the lower volume of production in that year. Profit (E04) also showed oscillation in the period, with the best result obtained in 2018. Research and development expenditure (E05) followed the trend observed in the profit indicator. In a while, overall equipment effectiveness (E07) began to be monitored in 2016 and improved in consecutive years. Market share (E06) is not monitored by the company due to the wide variety of products, which complicates competition assessment.

BR2 information was available in six economic indicators. Investment monitoring of research and development (E05) was not conducted by the BR2 factory, as product development activities were executed in the Japanese corporation headquarters. Sales performance (E01), revenue (E02) and costs (E03) indicated the same behavior, with the worst result achieved in 2015, followed by continuous increases in the following three years. Profit (E04) in BR2 grew until 2016, when it reached its best result, and a dramatic drop in 2017 and 2018. The market share figures (E06) indicated relative stability in the five years analyzed. Overall equipment effectiveness (E07) was constant in the period.

The economic analysis of IT1 revealed a slight oscillation in sales volume (E01) in the period, with alternating rise and fall, but maintaining the level of two thousand pieces sold per year. Revenues (E02) had the best result in 2014, but presented consecutive declines in the following two years, a scenario that was reversed with increases in 2017 and 2018. Operating costs (E03) were reduced by almost 23% in 2016 compared to the 2014 result, maintaining the level in the following two years. Profit (E04) had the worst result in 2015, but the scenario was reversed in 2016 when it reached the best level and remained with slight fluctuations in the following years. Research and development activities (E05) of products are conducted in a Technological Center of the corporation, outside the IT1 factory, for this reason there is no monitoring of this indicator. Another indicator without monitoring was market share (E06), which the manager justified by the wide variety of products, which makes it difficult to identify existing competitors in the market. The collection of economic data was finalized with the measurement of Overall Equipment Effectiveness (E07), which remained above the target of 85% in the five years.

The IT2 evaluation was performed in four economic indicators that were monitored by the factory. The results of sales volume (E01) and revenue (E02) showed a significant increase in 2018 compared to the level achieved in the previous four years. This positive phenomenon occurred due to the start of supply of products to a new customer from 2018. Positive result was also obtained in continuous reduction of operating costs (E03). Unlike other economic indicators that showed stability between 2014 and 2017, profit results (E04) showed marked fluctuations between in the period analyzed. Three economic indicators were not monitored by IT2: Research and development activities of products (E05) were conducted outside the IT2 factory, for this reason there was no monitoring of this indicator; the absence of the market share indicator (E06) was justified by the complexity in identifying its competitors; the overall equipment effectiveness indicator (E07) was still in its implementation phase.

In the environmental dimension, a distinctive processing of information is required to interpret the results, due to the consumption of resources being directly related to the volume of production. Thus, the absolute data were converted to values relative to the production volume.

The investigation in BR1 data showed the absence of monitoring of seven indicators. The control of the use of non-renewable (N01), renewable (N02) and hazardous (N04) materials was not performed by BR1. Additionally, studies related to life cycle design (N03) were performed, but without monitoring records. Furthermore, the company had initiatives for conservation of natural habitat (N09) that were punctual and uncontrolled. Then, the environmental performance of BR1 was calculated with data from seven indicators. The control of industrial effluents (N11) consisted of specifications of the liquid that was discarded, devoid of volume control. The evaluation of suppliers (N14) was performed by the headquarters and related data was not shared with BR1. Relative data from BR1 indicated that recycling volumes (N05) and reuse (N06) were increased from 2014 to 2017, with a small drop in 2018. The indicators of energy consumption (N07), emission of polluting gases (N10) and solid waste generation (N12) indicated that the best results occurred in years with the highest production volumes, 2014 and 2018, and the worst results in the year of lowest production, 2016. Water use records (N08) showed fluctuations in the period, with the best result in 2018 and the worst in 2014.

Data collection in BR2 was lacking monitoring in four indicators. Studies related to life cycle (N03) and supplier evaluation (N14) are conducted by the headquarters, without the disclosure of information to the factories. The reuse (N06) of material is not practiced by BR2. Another indicator not measured was the conservation of natural habitat (N09), due to the actions being punctual and uncontrolled. Thus, ten indicators were used to calculate the environmental performance of BR2. The consumption of non-renewable materials (N01) increased in 2017 and 2018, which manifested in reduction in the use of renewable materials (N02) in the same biennium. The best result in terms of replacing non-renewable materials with renewables was in 2016. The consumption of hazardous materials (N04) increased from 2014 to 2017, with a reduction in these inputs in 2018. The relative volumes of recycling (N05) and industrial effluent (N11) showed fluctuations, with the best results in 2014 and the worst in 2016. Additionally, energy (N07) and water (N08) consumption, emission of polluting gases (N10) and solid waste (N12) showed the best results in 2018. Compliance with environmental legislation (N14) occurred categorically in the period, with no fines from regulatory agencies.

The environmental analysis of IT1 showed the absence of monitoring of seven indicators. The distinction of materials from non-renewable (N01) or renewable (N02) sources used in the manufacture of products is not made by IT1. Furthermore, the factory does not monitor or receive information from the Technology Center on life cycle projects (N03). Material reuse (N06) is not applicable to IT1. Furthermore, there are specific actions of IT1 for the conservation of natural habitat (N09), but without monitoring of results. Furthermore, there is a deficit of monitoring of the volume of generated industrial effluents (N11). The evaluation of suppliers (N14) is conducted by the corporate purchasing department, which includes suppliers from all units of the group, for this reason, there is no indicator of monitoring of the specific suppliers of the investigated plant. The results of IT1 indicated a reduction in the use of hazardous materials (N04) over the years. With regard to recycling (N05), it was noted that the highest volume was in 2014 with declines in 2015 and 2016, an increase in 2017 and the worst level in 2018. Energy consumption (N07) and water (N08) had similar performances, with fluctuations in the period that revealed in 2015 the best result and in 2017 the worst. The emission of polluting gases (N10) reached the lowest level in 2015, with fluctuations in the following years. The amount of solid waste (N12) was continuously reduced in the analyzed period. Compliance with environmental legislation (N13) occurred fully during the period analyzed.

Environmental data on IT2 were available in twelve of the fourteen indicators used in this investigation. Recycling (N05) and reuse (N06) monitoring started in 2016. Natural habitat conservation (N09) and supplier assessment (N14) metrics were not considered by the company.

In the social aspect, BR1 information was collected except for three metrics. Employee training (S06) was conducted without counting hours dedicated to the activity. Additionally, investments in social projects (S08) took place on a one-off basis, without monitoring. Furthermore, the evaluation of suppliers (S09) did not include social requirements. The monitored social indicators of BR1 revealed an 11% reduction in the number of employees (S01) from 2014 to 2018. The smallest number of suggestions for improvements (S02) was recorded in 2016, the year in which the largest reduction in employees occurred. The increase in job satisfaction (S03) was measured by the organizational climate survey, which is conducted every two years. The number of lost working days (S04) was reduced to zero in the last two years, which denoted that the accidents (05) recorded were not serious. The gender equity indicator (S07) showed an increase in the proportion of women employed by the company. Additionally, the amount of customer complaint (S10) fell by 70% from 2014 to 2018. Compliance with social legislation (S11) was fully conducted by BR1.

In BR2, the lack of monitoring of three indicators was also noted. Gender equity (S07) is not controlled by the factory. Additionally, there is no planned budget for investments in social projects (S08), and vendor valuation (S09) does not contain social requirements. The social indicators of BR2 recorded a 13% reduction in the number of employees (S01) employed from 2014 to 2018. The number of suggestions for improvements (S02) has been increased over the years, but the numbers are low in proportion to the number of employees. The increase in job satisfaction (S03) was measured through the organizational climate survey conducted annually. Additionally, lost working days (S04) and number of accidents (S05) registered a slight increase in the five-year period. Employee training (S06) was intensified in 2017 and 2018, as an action to reduce the amount of complaints made by automakers, which is the metric that monitors customer satisfaction (S10). In terms of social legislation (S11), labor laws were fully met by BR2.

The social information of IT1 was collected in nine indicators. The metrics of job satisfaction (S03) and supplier evaluation (S09) were not monitored by IT1. The number of employees (S01) was reduced by 118 people in five years. The number of suggestions for improvements (S02) increased by 26% from 2014 to 2018. Lost working days (S04) due to medical leave showed oscillation over the period, with the best result in 2016. Number of accidents (S05) and number of hours of employee training (S06) also had ups and downs, with the lowest amount recorded in 2018. The gender equity indicator (S07) showed that the proportion of women employed in relation to men did not show significant changes in the period analyzed. Investments in social projects (S08) increased since 2015, reaching the best level in 2018. The customer satisfaction index (S10) is monitored by the number of complaints from vehicle manufacturers, which have reduced. Regarding compliance with social legislation (S11), the IT1 factory fully complied with the legal requirements in the period analyzed.

The social data of IT2 were monitored by eight of the eleven indicators consulted. The metrics of job satisfaction (S03), investments in social projects (S08) and supplier evaluation (S09) were not monitored by the factory. The indicator of improvement suggestions (S02) was implemented in 2018 and the customer satisfaction measurement criterion (S10) was modified in 2016, for this reason the previous years were not considered. The best social results were achieved in 2018 in six metrics, number of employees (S01), number of suggestions for improvement (S02), lost working days (S04), number of accidents (S05), gender equity (S07) and compliance with social legislation (S11). Exception to the indicators hours of employee training (S06) and customer satisfaction (S10) had the best results in 2017. The economic, environmental and social results are shown in

Table 3. Economic, environmental and social results of companies BR1, BR2, IT1 and IT2.

Description		Code	Unit	BR1					BR2					IT1					IT2
				2014	2015	2016	2017	2018	2014	2015	2016	2017	2018	2014	2015	2016	2017	2018	2014	2015	2016	2017	2018
Sales	↑	E01	[×103 un]	12.411	11.948	9.453	10.781	12.578	6.089	5.037	5.545	6.039	6.769	1.949	2.126	2.075	2.152	2.076	330	329	379	374	512
Revenue	↑	E02	[×106 €]	243	240	214	238	279	435	374	428	485	565	330	311	274	291	301	409	431	436	422	583
Operating costs	↓	E03	[€/pç]	16	16	19	16	16	12	13	12	12	11	163	142	126	129	128	1198	1299	1118	1118	1096
Profit	↑	E04	[×106 €]	14	27	15	25	31	7	10	20	12	6	12	10	13	12	13	14	2	12	4	22
Research and development expenditure	↑	E05	[×106 €]	8	8	9	9	11	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Market share	↑	E06	[%]	-	-	-	-	-	15.3%	15.6%	15.5%	15.5%	15.4%	-	-	-	-	-	-	-	-	-	-
Overall Equipment Effectiveness (OEE)	↑	E07	[%]	-	-	89%	91%	92%	87%	87%	87%	87%	87%	86.44%	87.00%	87.10%	87.96%	87.13%	-	-	-	-	-
Consumption of non-renewable materials	↓	N01	[ton]	-	-	-	-	-	103.188	85.835	90.876	107.324	129.122	-	-	-	-	-	49.506	49.129	55.789	54.240	71.757
Consumption of renewable materials	↑	N02	[ton]	-	-	-	-	-	49.055	40.111	47.771	43.672	40.114	-	-	-	-	-	7.476	6.741	7.271	7.043	6.887
Design for life cycle	↑	N03	%	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	1	1	1	1	1
Hazardous raw material used	↓	N04	[ton]	-	-	-	-	-	2.441	2.500	2.921	3.272	3.042	2.734	1.939	2.130	1.956	1.322	693	700	665	599	782
Recycling	↑	N05	[ton]	1.171	1.331	2.409	2.659	2.748	10.409	8.169	7.664	8.910	9.462	4.450	3.936	3.605	3.893	3.513	-	-	760	865	1.167
Reuse	↑	N06	[ton]	12.993	16.609	49.675	56.260	52.972	-	-	-	-	-	-	-	-	-	-	-	-	6.511	6.170	5.721
Energy consumption	↓	N07	[MWh]	80.161	79.994	75.596	80.689	84.366	189.595	169.684	187.124	190.979	191.074	51.324	54.528	59.354	63.034	60.136	56.975	52.654	50.473	50.373	54.745
Water consumption	↓	N08	[×103 m3]	155	114	97	112	113	1.423	1.316	1.358	1.348	1.323	119	97	111	158	129	117	118	112	115	112
Natural habitat conservation	↑	N09	[%]	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Greenhouse Gases (GHG) emission	↓	N10	[tonCO2]	7.595	7.432	6.930	7.352	7.626	91.900	82.643	81.988	85.552	85.445	19.680	20.819	20.737	21.564	20.572	56.988	1.848	103	27	150
Effluents (wastewater)	↓	N11	[m3]	-	-	-	-	-	597.957	605.558	692.600	714.462	688.233	-	-	-	-	-	53.179	44.302	41.412	40.555	47.214
Solid waste	↓	N12	[ton]	3.324	3.685	3.757	3.574	4.192	11.938	9.292	8.194	9.023	9.493	7.239	5.949	5.786	5.909	4.941	7.915	7.134	7.534	7.301	7.282
Compliance with the environm. legislation	↑	N13	[%]	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%
Supplier assessment (environm.)	↑	N14	[%]	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Number of employees	↑	S01		2.621	2.494	2.215	2.232	2.327	3.611	3.466	3.395	3.548	3.115	1.981	1.938	1.917	1.898	1.863	800	804	799	811	814
Employee suggestions for improvements	↑	S02		1.692	1.479	1.114	1.562	1.500	23	29	34	39	40	2.701	3.548	3.175	3.142	3.417	-	-	-	-	343
Job satisfaction	↑	S03	[%]	75%	75%	75%	83%	83%	71%	73%	74%	74%	76%	-	-	-	-	-	-	-	-	-	-
Lost workdays	↓	S04		411	21	130	0	0	10	9	11	21	17	78	69	37	76	51	89	0	0	22	0
Number of accidents	↓	S05		13	12	10	14	11	4	6	6	8	8	12	7	5	9	3	1	0	0	1	0
Employee training	↑	S06	[h]	-	-	-	-	-	15.995	16.837	17.632	24.329	24.585	35.852	33.160	15.132	27.802	14.405	32	33	30	62	35
Gender equity	↑	S07	[%]	14%	14%	15%	16%	17%	-	-	-	-	-	8%	8%	8%	8%	8%	3%	3%	3%	4%	4%
Investment in social projects	↑	S08	[×103 €]	-	-	-	-	-	-	-	-	-	-	41	36	49	50	54	-	-	-	-	-
Supplier assessment (social)	↑	S09		-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Customer satisfaction (0 km claims)	↓	S10		279	349	130	99	81	0	0	1	3	0	92	54	40	31	18	-	-	6	0	15
Compliance with the social legislation	↑	S11	[%]	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%	100%

.

After the consolidation of results, the next step was normalizing the data to convert the values of different dimensions into a dimensionless value. This procedure was performed using Equations (1) and (2). Data normalization facilitated the analysis of individual performance of the indicators, with ‘0’ being the worst and ‘1’ the best result. The normalized values of economic, environmental, and social results are presented in

Table 4. Data normalization of the economic, environmental and social indicators of BR1, BR2, IT1 and IT2.

Description		Code	BR1					BR2					IT1					IT2
			2014	2015	2016	2017	2018	2014	2015	2016	2017	2018	2014	2015	2016	2017	2018	2014	2015	2016	2017	2018
Sales	↑	E01	0.95	0.84	0.00	0.42	1.00	0.61	0.00	0.48	0.58	1.00	0.00	0.88	0.62	1.00	0.62	0.00	0.00	0.27	0.24	1.00
Revenue	↑	E02	0.45	0.40	0.00	0.37	1.00	0.32	0.00	0.28	0.58	1.00	1.00	0.66	0.00	0.31	0.49	0.00	0.12	0.15	0.07	1.00
Operating costs	↓	E03	1.00	0.88	0.00	0.91	0.98	0.58	0.00	0.25	0.52	1.00	0.00	0.57	1.00	0.90	0.94	0.50	0.00	0.89	0.89	1.00
Profit	↑	E04	0.00	0.74	0.04	0.65	1.00	0.08	0.26	1.00	0.39	0.00	0.60	0.00	1.00	0.79	0.88	0.61	0.00	0.49	0.08	1.00
Research and development expenditure	↑	E05	0.00	0.06	0.13	0.20	1.00	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Market share	↑	E06	-	-	-	-	-	0.00	1.00	0.67	0.67	0.33	-	-	-	-	-	-	-	-	-	-
Overall Equipment Effectiveness (OEE)	↑	E07	-	-	0.97	0.99	1.00	1.00	1.00	1.00	1.00	1.00	0.00	0.37	0.43	1.00	0.45	-	-	-	-	-
Consumption of non-renewable materials	↓	N01	-	-	-	-	-	0.79	0.76	1.00	0.49	0.00	-	-	-	-	-	0.00	0.07	0.25	0.48	1.00
Consumption of renewable materials	↑	N02	-	-	-	-	-	0.79	0.76	1.00	0.49	0.00	-	-	-	-	-	1.00	0.76	0.62	0.59	0.00
Design for life cycle	↑	N03	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	1.00	1.00	1.00	1.00	1.00
Hazardous raw material used	↓	N04	-	-	-	-	-	1.00	0.32	0.11	0.00	0.66	0.00	0.64	0.49	0.64	1.00	0.04	0.00	0.62	0.87	1.00
Recycling	↑	N05	0.00	0.11	1.00	0.95	0.77	1.00	0.73	0.00	0.28	0.05	1.00	0.27	0.08	0.20	0.00	-	-	0.87	1.00	0.98
Reuse	↑	N06	0.00	0.08	1.00	0.99	0.75	-	-	-	-	-	-	-	-	-	-	-	-	1.00	0.96	0.65
Energy consumption	↓	N07	1.00	0.85	0.00	0.33	0.84	0.47	0.01	0.00	0.38	1.00	0.81	1.00	0.19	0.00	0.09	0.00	0.20	0.60	0.58	1.00
Water consumption	↓	N08	0.00	0.85	0.63	0.59	1.00	0.42	0.00	0.25	0.58	1.00	0.43	1.00	0.70	0.00	0.41	0.02	0.00	0.44	0.36	1.00
Natural habitat conservation	↑	N09	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Greenhouse Gases (GHG) emission	↓	N10	0.96	0.88	0.00	0.40	1.00	0.35	0.00	0.43	0.59	1.00	0.00	1.00	0.34	0.24	0.61	0.00	0.97	1.00	1.00	1.00
Effluents (wastewater)	↓	N11	-	-	-	-	-	1.00	0.18	0.00	0.25	0.87	-	-	-	-	-	0.00	0.39	0.75	0.76	1.00
Solid waste	↓	N12	1.00	0.69	0.00	0.51	0.50	0.00	0.21	0.87	0.84	1.00	0.00	0.69	0.69	0.73	1.00	0.00	0.24	0.42	0.46	1.00
Compliance with the environm. legislation	↑	N13	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
Supplier assessment (environm.)	↑	N14	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Number of employees	↑	S01	1.00	0.69	0.00	0.04	0.28	1.00	0.71	0.56	0.87	0.00	1.00	0.64	0.46	0.30	0.00	0.07	0.33	0.00	0.80	1.00
Employee suggestions for improvements	↑	S02	1.00	0.63	0.00	0.78	0.67	0.00	0.35	0.65	0.94	1.00	0.00	1.00	0.56	0.52	0.85	-	-	-	-	1.00
Job satisfaction	↑	S03	0.00	0.00	0.00	1.00	1.00	0.00	0.40	0.60	0.60	1.00	-	-	-	-	-	-	-	-	-	-
Lost workdays	↓	S04	0.00	0.95	0.68	1.00	1.00	0.92	1.00	0.83	0.00	0.33	0.00	0.22	1.00	0.05	0.66	0.00	1.00	1.00	0.75	1.00
Number of accidents	↓	S05	0.25	0.50	1.00	0.00	0.75	1.00	0.50	0.50	0.00	0.00	0.00	0.56	0.78	0.33	1.00	0.00	1.00	1.00	0.00	1.00
Employee training	↑	S06	-	-	-	-	-	0.00	0.10	0.19	0.97	1.00	1.00	0.87	0.03	0.62	0.00	0.06	0.09	0.00	1.00	0.16
Gender equity	↑	S07	0.00	0.00	0.33	0.67	1.00	-	-	-	-	-	0.79	0.84	1.00	0.77	0.00	0.03	0.00	0.34	0.60	1.00
Investment in social projects	↑	S08	-	-	-	-	-	-	-	-	-	-	0.28	0.00	0.71	0.77	1.00	-	-	-	-	-
Supplier assessment (social)	↑	S09	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Customer satisfaction (0 km claims)	↓	S10	0.26	0.00	0.82	0.93	1.00	1.00	1.00	0.67	0.00	1.00	0.00	0.51	0.70	0.82	1.00	-	-	0.60	1.00	0.00
Compliance with the social legislation	↑	S11	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00

.

After data normalization, the weighted values of the companies’ results were calculated using Equations (5)–(7). The global economic index (ECO_global) was obtained by adding the weighted values of the seven economic indicators. BR1 data indicated significant economic growth from 2017, after the worst result of the period, registered in 2016. BR2 performance was also positive, with gradual and continuous results increasing. IT1 data showed that the 2014 result was the worst in the analyzed period. From 2015, economic performance steadily improved to peak in 2017, followed by a decline in 2018. Regarding IT2 data, the weighted values indicated that the best global economic result was achieved in 2018. However, this result would be better if there were monitoring of research and development metrics (E05), market share (E06) and global equipment effectiveness (E07).

The global environmental index (ENV global) showed that although the best result of the BR1 environmental index occurred in the year of highest production, 2018, the fluctuation of the global environmental values of BR1 revealed periods in which the efficiency of the use of resources and emission of pollutants was not proportional to the amount of parts produced. This finding was evidenced when comparing the results of 2014 and 2017. Production in 2017 was 13% lower than the volume produced in 2014, while the result of the environmental index in 2017 was 20% higher than in 2014. This phenomenon was also noticed in the environmental performance of BR2. The number of parts produced in 2014 and 2017 was approximately the same. However, the environmental index showed that the result of BR2 in 2014 was 35% higher than in 2017. In a five-year analysis, the IT1 operation achieved the best environmental result in 2015, with declines in the following two years and recovery in 2018. The increase in energy and water consumption reflected the weak environmental result in 2017. In the following year, the reversal of the scenario was conducted due to environmental gains with the reduction of the use of hazardous materials, water consumption and generation of solid waste. The weighted environmental values of IT2 indicated a significant increase in environmental results from 2015 when the company began to acquire electricity from green sources. The Italian electric power distribution company allows consumers to choose the source of power generation, green (wind) or conventional (thermoelectric). The taxation on wind power is EUR 0.02 more expensive per MWh, which has led more than EUR 1,100 in expenses per year. However, the choice of the plant for green energy resulted in a drastic drop in GHG emissions count, which improved the company’s environmental result. Another factor that positively influenced was the start of the recycling and reuse count of materials from 2016.

In the social dimension, the results of the global social index (SOC global) of BR1 denoted continuous advances in the social development of the company. The evolution of the social index was obtained by improving the organizational climate, increasing the proportion of women employed and the reductions in customer complaints and accidents. The social performance of BR2 was stable in the period, with a small decrease in 2017 followed by slight growth in 2018. The stability noted in the social index of BR2 was a consequence of the compensation of underperformances in some indicators by positive results in other metrics. In this sense, the social losses caused by the reduction of employees, increased days of leave and accidents, were compensated by the benefits obtained with the more pleasant organizational climate and the increases in the participation of employees in suggestions for improvements and hours of training. The social performance of IT1 increased in 2015 and remained stable, with a slight oscillation in the following years. With a different trend, the weighted values of IT2 revealed continuous improvement of social performance in the analyzed period. The weighted values of the indicators are shown in

Table 5. Weighted values and global economic, environmental and social indexes of BR1, BR2, IT1 and IT2.

Description		Code	Weight	BR1					BR2					IT1					IT2
				2014	2015	2016	2017	2018	2014	2015	2016	2017	2018	2014	2015	2016	2017	2018	2014	2015	2016	2017	2018
Sales	↑	E01	0.031	0.03	0.03	0.00	0.01	0.03	0.02	0.00	0.01	0.02	0.03	0.00	0.03	0.02	0.03	0.02	0.00	0.00	0.01	0.01	0.03
Revenue	↑	E02	0.033	0.02	0.01	0.00	0.01	0.03	0.01	0.00	0.01	0.02	0.03	0.03	0.02	0.00	0.01	0.02	0.00	0.00	0.01	0.00	0.03
Operating costs	↓	E03	0.037	0.04	0.03	0.00	0.03	0.04	0.02	0.00	0.01	0.02	0.04	0.00	0.02	0.04	0.03	0.03	0.02	0.00	0.03	0.03	0.04
Profit	↑	E04	0.038	0.00	0.03	0.00	0.02	0.04	0.00	0.01	0.04	0.01	0.00	0.02	0.00	0.04	0.03	0.03	0.02	0.00	0.02	0.00	0.04
Research and development expenditure	↑	E05	0.034	0.00	0.00	0.00	0.01	0.03	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Market share	↑	E06	0.018	-	-	-	-	-	0.00	0.04	0.03	0.03	0.01	-	-	-	-	-	-	-	-	-	-
Overall Equipment Effectiveness (OEE)	↑	E07	0.037	-	-	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.04	0.00	0.01	0.02	0.04	0.02	-	-	-	-	-
		ECO_global		0.08	0.10	0.04	0.13	0.21	0.09	0.09	0.14	0.14	0.15	0.06	0.08	0.11	0.14	0.12	0.04	0.004	0.07	0.05	0.14
Consumption of non-renewable materials	↓	N01	0.039	-	-	-	-	-	0.03	0.03	0.04	0.02	0.00	-	-	-	-	-	0.00	0.00	0.01	0.02	0.04
Consumption of renewable materials	↑	N02	0.035	-	-	-	-	-	0.03	0.03	0.03	0.02	0.00	-	-	-	-	-	0.03	0.03	0.02	0.02	0.00
Design for life cycle	↑	N03	0.039	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	0.04	0.04	0.04	0.04	0.04
Hazardous raw material used	↓	N04	0.032	-	-	-	-	-	0.03	0.01	0.00	0.00	0.02	0	0.02	0.016	0.02	0.032	0.00	0.00	0.02	0.03	0.03
Recycling	↑	N05	0.034	0.00	0.00	0.03	0.03	0.03	0.03	0.03	0.00	0.01	0.00	0.034	0.009	0.003	0.007	0	-	-	0.03	0.03	0.03
Reuse	↑	N06	0.036	0.00	0.00	0.04	0.04	0.03	-	-	-	-	-	-	-	-	-	-	-	-	0.04	0.03	0.02
Energy consumption	↓	N07	0.036	0.04	0.03	0.00	0.01	0.03	0.02	0.00	0.00	0.01	0.04	0.029	0.036	0.007	0	0.003	0.00	0.01	0.02	0.02	0.04
Water consumption	↓	N08	0.033	0.00	0.03	0.02	0.02	0.03	0.01	0.00	0.01	0.02	0.03	0.014	0.033	0.023	0	0.014	0.00	0.00	0.01	0.01	0.03
Natural habitat conservation	↑	N09	0.038	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Greenhouse Gases (GHG) emission	↓	N10	0.036	0.03	0.03	0.00	0.01	0.04	0.01	0.00	0.02	0.02	0.04	0	0.036	0.012	0.009	0.022	0.00	0.04	0.04	0.04	0.04
Effluents (wastewater)	↑	N11	0.031	-	-	-	-	-	0.03	0.01	0.00	0.01	0.03	-	-	-	-	-	0.00	0.01	0.02	0.02	0.03
Solid waste	↓	N12	0.033	0.03	0.02	0.00	0.02	0.02	0.00	0.01	0.03	0.03	0.03	0	0.023	0.023	0.024	0.033	0.00	0.01	0.01	0.02	0.03
Compliance with the environm. legislation	↑	N13	0.030	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03
Supplier assessment (environm.)	↑	N14	0.028	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
		ENV_global		0.13	0.15	0.12	0.16	0.20	0.23	0.13	0.16	0.17	0.22	0.11	0.19	0.11	0.09	0.13	0.10	0.16	0.29	0.31	0.37
Number of employees	↑	S01	0.013	0.01	0.01	0.00	0.00	0.00	0.01	0.01	0.01	0.01	0.00	0.01	0.01	0.01	0.00	0.00	0.00	0.00	0.00	0.01	0.01
Employee suggestions for improvements	↑	S02	0.026	0.03	0.02	0.00	0.02	0.02	0.00	0.01	0.02	0.02	0.03	0.00	0.03	0.01	0.01	0.02	-	-	-	-	0.03
Job satisfaction	↑	S03	0.029	0.00	0.00	0.00	0.03	0.03	0.00	0.01	0.02	0.02	0.03	-	-	-	-	-	-	-	-	-	-
Lost workdays	↓	S04	0.018	0.00	0.02	0.01	0.02	0.02	0.02	0.02	0.02	0.00	0.01	0.00	0.00	0.02	0.00	0.01	0.00	0.02	0.02	0.01	0.02
Number of accidents	↓	S05	0.038	0.01	0.02	0.04	0.00	0.03	0.04	0.02	0.02	0.00	0.00	0.00	0.02	0.03	0.01	0.04	0.00	0.04	0.04	0.00	0.04
Employee training	↑	S06	0.031	-	-	-	-	-	0.00	0.00	0.01	0.03	0.03	0.03	0.03	0.00	0.02	0.00	0.00	0.00	0.00	0.03	0.00
Gender equity	↑	S07	0.027	0.00	0.00	0.01	0.02	0.03	-	-	-	-	-	0.02	0.02	0.03	0.02	0.00	0.00	0.00	0.01	0.02	0.03
Investment in social projects	↑	S08	0.022	-	-	-	-	-	-	-	-	-	-	0.01	0.00	0.02	0.02	0.02	-	-	-	-	-
Supplier assessment (social)	↑	S09	0.031	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Customer satisfaction (0 km claims)	↓	S10	0.026	0.01	0.00	0.02	0.02	0.03	0.03	0.03	0.02	0.00	0.03	0.00	0.01	0.02	0.02	0.03	-	-	0.02	0.03	0.00
Compliance with the social legislation	↑	S11	0.034	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03	0.03
		SOC_global		0.09	0.10	0.11	0.14	0.18	0.13	0.13	0.13	0.12	0.15	0.11	0.16	0.16	0.14	0.15	0.04	0.10	0.11	0.13	0.16

.

The economic, environmental and social results collected from the companies were input data for the evaluation of sustainability in manufacturing, which is presented in the next section.

5.2. Sustainability Performance Assessment

The five-year analysis of the economic, environmental and social results of factories made it possible to evaluate the sustainability performance of the operations in relation to the history of their results and compared to other companies. Additionally, the isolated analysis of the average values of economic, environmental, and social performance allowed acknowledging the dimension that had the best result in the period. The findings of this analysis evidenced the direction of the actions implemented by the companies.

5.2.1. Sustainability Performance

The sustainability of the factories was calculated by adding the economic (ECO global), environmental (ENV global) and social (SOC global) indices of the factories. The information of the four companies evaluated in this study is presented in

Table 6. Results of company performance indexes.

Firm	Index	2014	2015	2016	2017	2018	Average Performance
BR1	ECO_global	8%	10%	4%	13%	21%	11%
	ENV_global	13%	15%	12%	16%	20%	15%
	SOC_global	9%	10%	11%	14%	18%	12%
	Sustainability	30%	35%	28%	43%	59%	39%
BR2	ECO_global	9%	9%	14%	14%	15%	12%
	ENV_global	23%	13%	16%	17%	22%	18%
	SOC_global	13%	13%	13%	12%	15%	13%
	Sustainability	45%	35%	43%	42%	52%	43%
IT1	ECO_global	6%	8%	11%	14%	12%	10%
	ENV_global	11%	19%	11%	9%	13%	13%
	SOC_global	11%	16%	16%	14%	15%	14%
	Sustainability	27%	43%	39%	37%	41%	37%
IT2	ECO_global	4%	0,4%	7%	5%	14%	6%
	ENV_global	10%	16%	29%	31%	37%	25%
	SOC_global	4%	10%	11%	13%	16%	11%
	Sustainability	18%	26%	47%	49%	67%	41%

.

The evaluation of the results was performed based on the performance scale containing three levels: low, regular and high. The definition of sustainability levels was referenced in the scale used by Ghadimi et al. [38], which set the low performance level from 0% to 33%, regular from 34% to 66% and high from 67% to 100%. Thus, the proportion was used to define the levels of the economic, environmental and social dimensions.

The strong sustainability quantified in this study consists of 23% of the economic dimension, 48% of the environmental and 29% of the social dimension. Then, the application of the scale in the dimensions of strong sustainability suggests the limits presented in

Table 7. Level of performance.

Dimension	Strong Sustain. Share	Level of Performance
		Low	Medium	High
ECO global	23%	≤7%	8 to 15	≥6%
ENV global	48%	≤16%	17 to 31	≥32%
SOC global	29%	≤9%	10 to 19	≥20%
Sustainability	100%	≤33%	34 to 66	≥67%

. The economic result is low up to 7%, regular between 8% and 15% and high from 16%. The environmental is low up to 16%, regular between 17% and 31% and high from 32%. The social is low up to 9%, regular between 10% and 19% and high from 20%.

The sustainability performance analysis of the factories indicated that the four cases are positioned at the regular level of sustainability. Despite the proximity of the average values of performance measured in five years, different trends were identified in the results of the companies. The performance information for the BR1, BR2, IT1 and IT2 factories is illustrated in Figure 5.

The economic performance of BR1 was regular in the first two years analyzed, fell to the low performance level in 2016, but had a sharp turnaround the following year and reached a high performance level in the economic indicator in 2018. The drop in the economic result of BR1 in 2016 was influenced by the unfavorable economic situation in Brazil, which recorded negative GDP in 2015 and 2016. The decline in the Brazilian domestic market associated with the devaluation of the local currency against USD stimulated exportation, which increased the economic gain of BR1. In environmental terms, BR1 data indicated low performance in the first four years analyzed. The level of performance was regular in 2018. The recovery in environmental results was driven by the increase of remanufacturing (reuse) of parts, efficiency in the use of energy and water and the reduction of emissions of polluting gases. In the social dimension, performance was continuously increasing, categorized as regular since 2015. The concern to improve the social aspect was noted in the information collected from BR1. Actions to prevent accidents, make the work environment more pleasant and raise the quality of products directed the social result of BR1 close to the high performance level in 2018. The sustainability index of BR1 reflected the positive results achieved in the last two years analyzed. The economic performance was emphasized in the change in the level of sustainability of BR1. In this sense, the economic advance in the 2017–2018 biennium was superior to the evolution of the environmental and social dimensions added together.

The results of BR2 indicated a regular level of economic performance in the analyzed period. The gradual development of BR2 drove the 2018 result close to the high performance level. Increased sales and reduced costs contributed to this level. Product exports increased in the second half of 2016. Furthermore, the cost reduction that involved the reduction of the purchase of renewable raw materials were strategies defined by the management. The environmental performance of BR2 was regular, except in 2015 when it had a low performance level, motivated by increased energy and water consumption and emission of polluting gases. The negative environmental results of BR2 in 2015 were influenced by the drop in product sales of that year. The economic aspect had a direct impact on the environmental result of BR2 over the years. The focus on cost reduction decreased the volume acquired renewable raw material, which aggravated the result of this indicator. On the other hand, actions to cut costs have contributed to the lowest numbers in water electricity consumption. The social data of BR2 indicated stability of the results over the years. Progress in some indicators compensated for the regression in other metrics. In this respect, the cost reduction negatively influenced social performance results, with the reduction of employees. On the other hand, actions that did not require investment contributed to improving the organizational climate and training operators and analysts in jobs. The values of the BR2 sustainability index were at the regular level during the five years. The worst result was in 2015, due to the drop in environmental performance. Actions with the purpose of reducing costs contributed to increase the sustainability in the company.

The economic results of IT1 revealed poor performance in 2014, with consecutive increases in the following three years, a small drop in 2018, but with maintenance at the regular level of performance. The efforts made to reduce costs, associated with the maintenance of sales volume and profit, have been important for the economic growth of the business. The environmental performance of IT1 showed oscillation in the analyzed period. Predominantly, the operation had low performance ranging from 9% to 13%, except for the 2015 result that was the best of the period and reached the regular level. The level increase achieved in 2015 was due to the reduction of electricity and water consumption and emission of polluting gases. In contrast, the drop in performance in 2016 was motivated by the increased use of hazardous materials, energy, water, emission of polluting gases and reduced recycling. The social result of IT1 was regular in the five years analyzed. The 2015 result showed an increase of 45% compared to the previous year. This phenomenon was mainly due to the increase in the number of considered suggestions for improvements, the reduction of accidents and complaints from vehicle manufacturers, which increased the result of customer satisfaction. Although the number of employees has decreased over the years, this indicator is the one that has the least relevance in the social dimension. Additionally, the gains obtained in some metrics stood out to the losses and ensured the global social result with a slight oscillation in the last four years. As a consequence of the economic, environmental and social results, the sustainability performance of IT1 showed variations in the period. Initially, the sustainability of IT1 presented low performance, but changed levels from 2015 when it obtained the best result, influenced by the environmental gains recorded. Still with regular performance, decreases were noted in the following two years, until recovery in 2018.

The economic data of IT2 revealed low performance in the period from 2014 to 2017, with evolution to the regular level in 2018. The 2015 result was the lowest of the period, with the worst sales, cost and profit figures. In contrast, the jump achieved in 2018 was driven by a new customer, which boosted sales and revenues by 38%. Additionally, profit in 2018 was 270% higher than the average result recorded between 2014 and 2017. The environmental performance of IT2 has continuously evolved in the period, starting in the low performance level in 2014, followed by three regular years, attaining high performance level in 2018. The change in level in 2015 was due to the start of electricity acquisition from green sources, which drastically reduced the GHG emission count. Another significant increase in environmental performance occurred in 2016, when IT2 began monitoring recycling and reuse indicators. Furthermore, actions implemented to reduce the use of hazardous materials, electricity, water, generation of industrial effluent and solid waste have ensured continuous gains for the environment. The social results of IT2 also grew continuously in the period. Social indicators recorded low performance in 2014, a result that was raised to the regular level in the following years. The improvement noted in 2015 was due to reductions in lost working days and accidents. In 2016, the performance increase was caused by the start of a new customer complaint counting methodology. The following year, increased employee numbers, training hours, and customer satisfaction leveraged social performance. In 2018, IT2 began counting the number of suggestions for improvement, which contributed to raising the global social result of the company.

The sharp growth in IT2 sustainability performance indicated that the actions taken by managers facilitated economic gains and benefited environment and society. The relative values showed an increase of 360% of IT2 sustainability in five years. The sustainability result in 2018 was higher than the average value in five years, which denoted the adequacy of strategic planning in the definition of actions to lead the company to the high performance level of sustainability.

5.2.2. Individual Analysis of Sustainability Dimensions

The participation of the dimensions in strong sustainability limited economic performance by 23%, the environmental by 48% and the social performance by 29%. The scale that prioritized ecosystem conservation provided the presentation of consolidated data in the sustainability index, which offered the global view of the factory’s results. However, the grouping of information made it difficult to understand the individual performance of the dimensions.

In this context, the average performances of the sustainability dimensions were investigated individually. The analysis consisted of converting the dimension data into a scale from zero to one hundred percent. The operation consisted of dividing the average value of the dimension in the sustainability indicator by the maximum value of the dimension in strong sustainability. Thus, the economic result was divided by 23%, the environmental by 48% and the social by 29%, which are the limits defined in strong sustainability. Performance data on an individual scale by dimension are shown in

Table 8. Analysis of sustainability dimensions.

Dimension	Strong Sustain. Share	Average Value in the Sust. Index				Analysis by Dimension (0–100%)
		BR1	BR2	IT1	IT2	BR1	BR2	IT1	IT2
Economic	23%	11%	12%	10%	6%	48%	52%	43%	26%
Environm.	48%	15%	18%	13%	25%	31%	38%	27%	52%
Social	29%	12%	13%	14%	11%	41%	45%	48%	38%

.

The findings of this analysis evidenced the direction of the actions implemented by the companies. The result of BR1 denoted that the economic dimension was predominant. The actions taken to leverage sales and profit from product exports were emphasized. The BR2 result also pointed out that economic growth was higher than the socio-environmental. With this, the analysis in Brazilian factories revealed that the economic aspect was a priority in decision-making. BR2’s strategy to reduce the acquisition of renewable raw materials to reduce costs is an example of this finding.

On the other hand, data from factories in Italy emphasized the social and environmental aspects. The IT1 result showed that the return obtained with the reduction of accidents, customer satisfaction, and local community, through social projects, were greater than economic growth. In relation to IT2, environmental issues were a priority. One evidence of this finding was the additional cost for acquiring green energy, which drastically reduced the GHG emission count. Data from the performance analysis of companies by dimension are shown in Figure 6.

6. Discussion

The research carried out raised significant aspects in sustainability assessment. Initially, no quantitative studies on strong sustainability were identified. Despite the growing interest in the subject, the publications on strong sustainability presented a qualitative approach. The absence of quantitative studies is justified by the difficulty in establishing the natural limits of resource supply and pollution absorption. This finding was also reported in the study by Bjørn and Røpke [21], who pointed out the need for an in-depth knowledge in environmental sciences to count ecosystem thresholds, which makes it difficult to approach strong sustainability in the corporate context. Despite the complexity in estimating the loading capacity of natural systems, the relative evaluation conducted in this research work has proven that it is possible to insert the principles of strong sustainability in the monitoring of the performance of manufacturing companies.

The definition of weights for economic, environmental and social indicators was also a significant result of this work. The analysis of experts to attribute the relevance of sustainability indicators provided the database for the calculation of weights of the monitored metrics. Survey with specialists was also conducted in the studies of Beekaroo et al. [110] and Chanmg and Cheng [111] who collected opinions from for the construction of their weighted models of sustainability assessment. The pairwise comparisons used to derive weights of indicators showed an easy method of integrating subjective judgment (human perception) and scientific calculation. This method enabled to determine the criteria weights indirectly based on scores of relative importance for strong sustainability. The pairwise comparisons calculation was simple and straightforward, which was an advantage of this procedure. A potential vulnerability of this method may be at converting qualitative opinion in quantitative scale, which could show biased conclusion. To mitigate this vulnerability, the survey should collect data from knowledge and experience of experts. The result of the expert analysis was consistent with the definition of strong sustainability of [17], which denoted the adequacy of this methodology.

In terms of application of the proposed model, the sustainability evaluation of the factories allowed the individual performance analyses of the cases and the comparison of their results. The four companies recorded a regular level of sustainability on the average in the analyzed period. However, the evolution of the results showed significant differences.

BR1 data demonstrated sustainable development, which was driven mainly by economic growth. Despite the unfavorable scenario of the Brazilian economy, the USD appreciation in relation to the local currency stimulated the increase in exports, which reflected in significant sustainable progress between 2017 and 2018. However, the economic result of BR1 in 2018 was close to the limit of the strong sustainability scale (23%), which indicated that the company needs to advance in the socio-environmental aspect to achieve the high performance level of sustainability. This finding corroborated the conclusions of Bishop et al. [55] that demonstrated the importance of social well-being and natural capital in strong sustainability development projects. In this sense, the implementation of indicators in metrics that are not controlled is the first step to improve environmental and social management.

BR2 results also emphasized sustainability developments. Actions focused on cost reduction contributed to economic and environmental growth. However, decisions that prioritized the financial result undermined the performance of some environmental indicators, such as the reduction of the use of raw materials from renewable sources. This evidence ratified the conclusions of Martins [20], who argued the existence of restrictions for strong sustainability due to the predominance of neoclassical economy in business, characterized by the linear flow of natural resources. In this context, studies on strong sustainability, Circular Economy and Industrial Ecology emphasize the gains made for sustainable development. Thus, the increase of research on these topics is fundamental to change the mindset of governments and managers to respond to the demands of business stakeholders. With this, the elaboration of incentive laws to intensify the circularity of products provides the sustainable development of industrial operations.

The mean value of the IT1 sustainability index in the period was the lowest among the four cases investigated. Furthermore, the average environmental result was the worst among factories. The low environmental performance performed by IT1 revealed that the company needs to prioritize actions in this area. The absence of indicators of consumption of non-renewable and renewable materials, life cycle project, reuse, conservation of natural habitat, generation of industrial effluent and evaluation of suppliers, lowered the company’s results. This finding corroborated the results of Oliveira Neto et al. [19] that considered these factors in the stock framework to promote strong sustainability in companies. The source control of raw material is possible, as evidenced in the cases of IT2 and BR2.

Additionally, the results of the monitored environmental indicators of IT1 revealed increased energy, water and greenhouse gas emissions. This finding denoted negligence in the definition of strategies and action to reverse the negative scenario. More specifically in the 2016–2017 biennium, environmental results fell and economic gain increased. The compensation noted in the period characterized weak sustainability, which restricted the progress of the sustainable development of the operation. The statement that weak sustainability is inappropriate for sustainable advancement was also mentioned by Lorek and Fuchs [52], who outlined the shortcomings of public policies related to consumption. Thus, the use of the sustainability assessment model by companies will enable managers to improve process control, with defined indicators and targets established by senior management, who will be responsible for the critical analysis of results at the end of the period.

Different from the scenario observed in IT1, the IT2 results showed growth of the three dimensions of sustainability. Concern about environmental issues encouraged action that provided high-level environmental performance in 2018. This level was reached due to reductions in consumption of non-renewable materials, hazardous materials, energy and water. Additionally, the reduction of greenhouse gas emissions, industrial effluent and solid waste also contributed to the best result in the analyzed period. The efficient use of resources associated with the reduction of air, water and soil pollution, noted in the IT2 result, were also findings reported in the study by Oliveira Neto et al. [19] as guidelines for sustainable development. With this, actions taken by IT2 managers demonstrated alignment with the principles of strong sustainability.

Additionally, IT2 actions in the social sphere provided significant gains that boosted the performance of this dimension. In this sense, suggestions for improvements began to be encouraged and monitored, which increased the involvement of employees in improving processes focused on quality, environment and occupational safety. Accident prevention was a priority for IT2, which reflected the number of occurrences measured in the period, two cases in five years. The preservation of the physical integrity of people was also pointed out by Oliveira Neto et al. [19] as a key factor for sustainable manufacturing, which is a guideline for promoting strong sustainability. Thus, the high performance level of sustainability of IT2 in 2018 demonstrated that the change of level requires sharp advances in the three dimensions of sustainability.

The performance analysis of the factories by size indicated that the economic growth was higher than the environmental and social gains in the units installed in Brazil. This finding denoted the reactive behavior of companies with social and environmental issues. The actions are directed to the control of pollutants and meet requirements, standards and legislation. On the other hand, Italian factories have shown concern about social and environmental aspects. Actions aimed at preventing accidents and customer complaints reflected in the social result of IT1, which was superior to economic and environmental results. Increased efficiency in the consumption of natural resources and the prevention of air pollution were fundamental to emphasize the environmental development of IT2.

The challenge for managers is to continuously increase economic gains, environmental preservation and social well-being. The sustainability assessment model developed in this study stimulates continuous improvement through the monitoring of the sustainability index, which provides an overview of the sustainable performance of the business.

The contribution of the developed model to the practice in operations management was ratified by the testimony of the managers of IT1, IT2 and BR2. The manager of IT1 emphasized that the model contributes to improving sustainability in the industry. Based on the results of this study, the IT1 manager recognized the potential to be explored and cited the example of industrial effluent treatment for water reuse. The head of IT2 was interested in the sustainability index calculation with the definition of indicators and the assignment of weights. Furthermore, the IT2 leader pointed out that the model offers a simple tool to measure achievements. The BR2 manager believes that the tool contributes to sustainability management, as it presents the overview of the progress achieved, and the indicator helps to identify actions to be prioritized among the existing possibilities to achieve better results. However, the BR2 manager emphasized that, despite being a good tool, it is unlikely that the use of the sustainability indicator will reduce budget constraints in implementing environmental improvements and occupational safety.

7. Conclusions

This research dealt with the evaluation of sustainability in manufacturing that incorporated the concept of strong sustainability in the measurement of the performance of industries. The analysis of expert opinions resulted in the weights of economic, environmental, and social indicators, from the perspective of strong sustainability. The application of the evaluation model in manufacturing companies showed that it is possible to insert strong sustainability in corporate management.

The sustainability assessment model is based on the methods used in operations management for process control. The calculation of the sustainability index is simple to be performed with the aid of a spreadsheet. However, data collection is complex and requires standardization to increase the accuracy of the results.

The investigations in companies showed that the achievement of the highest level of sustainability was only possible with growth in the three dimensions. The high level of sustainability achieved by IT2 in 2018 demonstrated that investments in environmental preservation and efficiency of use of natural resources does not inhibit economic growth. On the other hand, the strategy of prioritizing cost reduction returned financial gain to BR2, but impaired environmental performance in some indicators.

Another aspect observed in this research was that sustainability management in operations is performed intuitively based on scattered information from individual metrics. As a result, shareholders, companies and business stakeholders find sustainable developments comprehension problematic, on account of an absence of consolidated data in corporate reports.

In this sense, the practical contribution of this research consisted of offering an easy-to-operate tool, which helps the analysis of results for the establishment of strategies and unfolding of actions by Management. The statements collected from managers of companies focus of this study confirm the importance of the sustainability assessment model. Additionally, the disclosure of the sustainability index in corporate reports facilitates business progress comprehension for stakeholders.

The theoretical contribution of the research was the quantitative study of strong sustainability, which calculated the participation of the dimensions in strong sustainability. Another contribution to the literature was the development of the sustainability assessment model in manufacturing, with metrics weighted according to the relevance to strong sustainability. Furthermore, the study of multiple cases that compared factories in two countries showed distinct priorities in taking corporate action.

The methodology presented in this study is appropriate for application in other business sectors. However, it is noteworthy that the indicators used in this research were selected from models of sustainability assessment in manufacturing. For this reason, sustainability assessment in other sectors such as agriculture, power generation or civil construction requires analysis of adequacy of the proposed indicators, addition of area-specific metrics and analysis of recent approaches of sustainability dimensions such as institutional and cultural factors.

The difficulty encountered in defining the thresholds of the natural system in terms of resource supply and pollution absorption was a restriction for the evaluation of sustainability performance in absolute terms. Therefore, this research suggests conducting quantitative investigations to increase knowledge about the ecosystem’s loading capacity.

The assessment of sustainability indicators prompts the elaboration of methodology to standardize the collection and cataloging of economic, environmental, and social metrics, which will increase the quality and reliability of the reported information. Furthermore, the evaluation of sustainability in future case studies will contribute to the dissemination of strong sustainability in organizations.