Next Article in Journal
Alternative Communications about Sustainability Education
Next Article in Special Issue
Integrated Assessment of the Water, Sanitation and Hygiene Situation in Haitian Schools in the Time of Emergency
Previous Article in Journal
“Bring It under the Legacy Umbrella”: Olympic Host Cities and the Changing Fortunes of the Sustainability Agenda
Previous Article in Special Issue
Remotely Accessible Instrumented Monitoring of Global Development Programs: Technology Development and Validation
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Performance Analysis of the Capability Assessment Tool for Sustainable Manufacturing

1
School of Aerospace, Mechanical and Manufacturing Engineering, RMIT University, PO Box 71 Bundoora VIC 3083, Victoria 3083, Australia
2
School of Aerospace, Mechanical and Manufacturing Engineering, RMIT University, GPO Box 2476, Melbourne, Victoria 3001, Australia
3
Centre for Design, School of Architecture and Design, RMIT University, GPO Box 2476, Melbourne, Victoria 3001, Australia
*
Author to whom correspondence should be addressed.
Sustainability 2013, 5(8), 3543-3561; https://doi.org/10.3390/su5083543
Submission received: 4 July 2013 / Revised: 8 August 2013 / Accepted: 9 August 2013 / Published: 19 August 2013

Abstract

:
This paper explores the performance of a novel capability assessment tool, developed to identify capability gaps and associated training and development requirements across the supply chain for environmentally-sustainable manufacturing. The tool was developed to assess 170 capabilities that have been clustered with respect to key areas of concern such as managing energy, water, material resources, carbon emissions and waste as well as environmental management practices for sustainability. Two independent expert teams used the tool to assess a sample group of five first and second tier sports apparel and footwear suppliers within the supply chain of a global sporting goods manufacturer in Asia. The paper addresses the reliability and robustness of the developed assessment method by formulating the expected links between the assessment results. The management practices of the participating suppliers were shown to be closely connected to their performance in managing their resources and emissions. The companies’ initiatives in implementing energy efficiency measures were found to be generally related to their performance in carbon emissions management. The suppliers were also asked to undertake a self-assessment by using a short questionnaire. The large gap between the comprehensive assessment and these in-house self-assessments revealed the suppliers’ misconceptions about their capabilities.

1. Introduction

Industrial economics are increasingly dominated by major concerns about environmental factors, most notably climate change, the shortage of non-renewable energy sources, water and raw material supplies, and commodity price increases [1]. In response to these concerns, national and international authorities have established standards, protocols, regulations, and schemes, such as emissions trading and binding international targets for pollution control and resource efficiency, e.g., the United Nations Framework Convention on Climate Change (UNFCCC) [2]. Manufacturers that do not adapt to these environmental, social and economic changes will be affected in the emerging competitive market for sustainability [3]. Furthermore, the increasing public awareness of the social, environmental, and economic benefits associated with sustainable manufacturing, and of the risks, liabilities, and costs associated with unsustainable practices, represent additional drivers for a more fundamental shift towards sustainability within the manufacturing sector [2,3,4,5,6,7]. It is important to note that by adopting the sustainable manufacturing concept, companies have the potential to create new market opportunities and improve their corporate reputation [8,9]. This in essence supports the development of globally competitive manufacturing businesses [10,11] and positively impacts on current financial and market performance of such businesses [12].
Manufacturing involves continual management of a variety of inter-related disciplines and activities, including product design, raw material purchasing, production planning and processes, and environmental compliance [4]. To manage such operations effectively, complex decisions must be made, involving often conflicting requirements. Contrary to traditional manufacturing, which is typically dominated by decisions based on economics only, sustainable manufacturing management is based on a more comprehensive, multi-criteria, decision-making process, which seeks to balance social, e.g., [13], environmental and economic elements of manufacturing [2,14,15,16]. This is essential in order to minimize the likelihood that the solutions adopted today will not become problems in the future.
Criteria which are typically considered pertinent to environmental sustainability assessment include the following examples [15]:
  • Made from renewable materials;
  • Produced while making the most efficient use of resources such as water and energy;
  • Produced with minimal pollution to the environment and waste;
  • Capable of being washed in low temperatures using environmentally friendly agents (where applicable);
  • Capable of being returned safely to the environment at the end of their useful life.
This paper is scoped around the environmental aspects of sustainable manufacturing. Implementation of environmentally sustainable practices in manufacturing of sports apparel and sports footwear must be based on clearly-defined environmental targets for reduction in emissions, water consumption and waste generation. The materials and processes used in manufacturing are of key concern as they carry with them many potential environmental risks. For example, while many manufacturers state that they aim to increase the percentages of recycled materials in their products, instead, virgin materials are being adopted as the main raw materials for sports apparel. Also, particular manufacturing operations in sports apparel and footwear manufacturing—e.g., injection moulding, used in manufacturing of outsoles and heels—are energy intensive, which means that they account for high levels of emissions [17]. Furthermore, the demand for textile fibre worldwide is increasing along with the demand for high performance sports apparel. Performance apparel items impact on the environment at every stage of their life cycle as shown in Table 1. Sports apparel and footwear in general have a heavy carbon footprint, which results primarily from pollution, consumption of non-renewable resources and waste associated with manufacturing. Manufacturers are required to reduce or alleviate these impact areas across the entire supply chain. Whether they are able to reduce energy consumption and emissions, water consumption and waste largely depends on whether staff and management are capable of identifying and implementing the required improvements within their particular environments.
Table 1. Environmental impacts across the life cycle of sport apparel.
Table 1. Environmental impacts across the life cycle of sport apparel.
Lifecycle stageEnvironmental impact
Raw materials (growth, acquisition and processing)Resource consumption, greenhouse gas emissions, air/water pollution and toxicity, soil degradation/contamination, biodiversity/land use, solid and hazardous waste
Fibre production (natural and synthetic)Greenhouse gas emissions, air/water pollution and toxicity, soil degradation/contamination, biodiversity/land use
Clothing production and garment assemblyGreenhouse gas emissions, air/water pollution and toxicity, soil degradation/contamination, biodiversity/land use
PackagingSolid and hazardous waste
DistributionGreenhouse gas emissions
RetailSolid and hazardous waste
UseResource consumption, solid and hazardous waste
End of life managementGreenhouse gas emissions, solid and hazardous waste
Due to the relative newness of environmental sustainability in the manufacturing sector (often called green manufacturing) as a discipline [9], the industry needs to acquire new capabilities (knowledge, skills, and values) in order to convert environmentally-sustainable manufacturing aspirations into practice. Many global manufacturers have already set strategic environmental targets for themselves, e.g., [18]. Whilst setting and monitoring tangible environmental targets are essential steps towards practicing sustainability, the targets will not be achieved if a manufacturer does not possess the required capabilities that need to be identified strategically by using a comprehensive sustainability framework [3]. Such a framework needs to go beyond the environmental management systems traditionally practiced (which mainly encourage environmental awareness) [11], by including particular assessment approaches to identify the capability gaps that need to be addressed in order to achieve the set environmental sustainability targets and encourage suppliers to proactively take environmental initiatives [19]. Sustainability frameworks developed to date are typically focused on strategy and decision making processes, rather than capability assessment. De Bakker and Neijhof developed a framework for social aspects of sustainability [20]; however, no specific capabilities were proposed. Kinderytė reviewed a number of sustainability assessment frameworks [21]. One of these frameworks, the Sustainability Assessment for Enterprises (SAFE), developed by the Wuppertal Institute, is designed to assess a company’s performance on economic, ecological, social and communication metrics. One aspect of the SAFE framework is the identification of the qualification needs of employees [21]. The SAFE system, however, does not extend into the supply chain. Likewise, the Sustainability Competency and Opportunity Rating and Evaluation (SCORE) system [22] includes aspects of assessing capability across sustainability practices, but also does not extend this assessment to the supply chain. A common aspect of these frameworks is the need and importance of involving stakeholders in framework development [20,23], including the company, its customers and suppliers, and government agencies There are sustainable manufacturing frameworks and/or environmental assessment tools introduced by other researchers which are solely based upon some particular solutions only such as using cleaner energies, using materials with low environmental impacts, and using improved technologies to reduce environmental footprints—e.g., GHGs, and hazardous waste—associated with manufacturing processes [6,24]. This is while there are also examples of some frameworks and assessment tools (e.g., for environmental management performance) which have limited but broad objective-based capabilities—e.g., pollution reduction capability—along with some solution-based capabilities such as employing advanced technologies used as the basis to quantify (assess) the environmental management performance of manufacturing firms [25]. In sum, after reviewing existing frameworks, it was identified that there was a need to develop a new framework to assess particular capabilities across the supply chain that can be applied to a broad range of industries. Such a framework needs to be objective based, focusing on environmental footprints and the associated capabilities (at the manufacturers’ level) in order to provide a real value add to different types of manufacturers.
In an earlier article published by the authors [26], the development and implementation of a new sustainable manufacturing capability framework for an existing supply chain within the sports apparel and footwear industries was described in detail. This framework, which is primarily concerned with formulating critical sustainable manufacturing competencies, was used to establish a metrics assessment tool to evaluate capabilities relating to environmental sustainability at manufacturing level in order to identify capability gaps, improvement strategies, and training needs to achieve a set of environmental targets, i.e., reduction of energy and water usage, as well as the reduction of waste and emissions generation. Identifying such gaps is an important capability by itself that allows a manufacturer to realize its sustainability-related threats and opportunities and create new (dynamic) capabilities accordingly to keep pace with outside changes [27,28]—e.g., new environmental regulations and targets. The earlier paper published by the authors [26] mainly focused on using the framework and the associated assessment tool to identify such gaps and suggest training needs to fill the gaps. The present paper is dedicated to evaluating the performance, robustness, and reliability of this sustainable manufacturing framework and the metrics assessment tool built upon it. Although the framework developed as part of this research is focused on environmental sustainability at the manufacturing level, the approach used in this case can be extended beyond manufacturing to achieve a comprehensive system approach for sustainable manufacturing (i.e., life cycle approach) [8,16,24]. It is noteworthy that the present research was conducted in collaboration with a global sporting goods manufacturer, involving five of its first and second tier sports footwear and apparel suppliers in Asia, aimed at assisting the manufacturer to meet its strategic environmental targets across the supply chain.

2. Sustainable Manufacturing Capability Assessment Tool (CMAT)

The first step towards assessing the suppliers’ capabilities relating to sustainable manufacturing was to develop a sustainable manufacturing framework (SMF), (Figure 1). This framework was developed in consultation with the manufacturer and the participating suppliers, as well as by reviewing the sustainability targets and associated documents of the manufacturer and similar global manufacturers—e.g., [29] and other external resources from Australia’s National Training Information Service [30], the Australian Department of Education, Employment and Workplace Relations [31], and ISO standards [32,33]. The framework comprises of three levels termed clusters, applied outcomes, and capabilities.
Figure 1. Sustainable manufacturing framework (SMF) (Note: there are 170 capabilities under these eight clusters and 18 applied outcomes which have not been shown in this figure).
Figure 1. Sustainable manufacturing framework (SMF) (Note: there are 170 capabilities under these eight clusters and 18 applied outcomes which have not been shown in this figure).
Sustainability 05 03543 g001
The eight clusters in this framework reflect the environmental indicators and initiatives set by the manufacturer, similar to those also provided in corporate reports [34,35]. Environmental capabilities and the associated indicators [10,32,33,36,37] are broadly grouped within three management categories, including resources (energy, water, and material), emissions (waste and greenhouse gas emissions), and environment (e.g., ISO 14001 standards). These are the key elements that to some extent have also been considered by similar studies in these areas, e.g., Material; Energy; and Waste, MEW, used by Smith and Ball [38]. These eight clusters were grouped into three broader areas of sustainable manufacturing; resource management, emissions management, and improved management practices (in order to introduce environmental sustainability). The 18 applied outcomes relate to industry practices within the relevant environmental indicators and initiatives.
Based on the sustainable manufacturing framework (SMF), we have developed a comprehensive capability metrics assessment tool (CMAT), which was used to assess the suppliers’ capabilities and complete a needs analysis. The capabilities were assessed on a Likert Scale, ranging from one to five (unsatisfactory to excellent) based on the following examples of attainment:
  • Unsatisfactory (1): There is no capability at all.
  • Satisfactory (2): There are just enough, minimal or marginal capabilities.
  • Good (3): Developing capability above minimal or marginal, can do basic things independently.
  • Very good (4): Developed capability, can plan regular actions independently.
  • Excellent (5): Can lead changes, plan improvements, and grasp new techniques.
In order to ensure a fair, robust, and reliable assessment, the capabilities were assessed by two independent expert teams (with up to four personnel in each team), one team from RMIT University and one team from the manufacturer. Furthermore, four assessment methods were employed in the assessment process. Each core capability (specified based on the industry being assessed) was assessed using two of the four methods, while the remaining capabilities were assessed using at least one method. The four methods used in this assessment were:
  • Walk-through assessment, based on observing capability in work activities.
  • Written answers to one or more questions based on simulated workplace activities.
  • Interviews through responses to verbal questions.
  • Work samples (e.g., a project report) indicating prior demonstration of capability.
Different questions were used to assess each capability using each of the four methods described above. For example, the “reduce energy use” applied outcome under the “energy efficiency” cluster (see Figure 1) includes the following capabilities to be investigated:
  • Measure, record, and report energy use in factory operations.
  • Compare energy use records for operations to previous records and industry best practice.
  • Relate energy used in operation to total consumption for the factory.
  • Use rankings and identify energy losses to determine process which can be improved.
  • Document a flowchart showing energy flows for an operation in the factory.
  • Apply continuous improvement.
Each assessment method for a particular capability includes a few questions/items to be asked/checked against pre-defined evidence of attainment as a reference to avoid registering any possible expert teams’ perceptions about the participating suppliers. For example, under the “reduce energy use” applied outcome, the following open questions were used in an interview with the focus group:
  • Explain how you measure energy use for the factory (evidence: electricity bills, oil/diesel/coal bills, energy balances).
  • Explain how you measure or calculate energy use for a machine or a production line (evidence: direct measurement using tongue tester; estimation based on machine energy rating and time of use, energy balances).
  • Explain how you report energy use (evidence: the manufacturer’s Environmental Metrics Reporting Tool, own system; can be for all operations or machine-specific. Need to provide examples of internal or external reports).
The overall score of a cluster was calculated by following the procedure illustrated in Figure 2 and using the following equation:
Sustainability 05 03543 i001
Where “C” is the overall average score of the cluster (a number between 1 and 5) calculated by an assessment group i, “A” is the number of assessors; “N” is the number of applied outcomes under the cluster being assessed (1, 2, or 3 according to Figure 1); “Mn” is the number of capabilities under the applied outcome of “n” (n varies from 1 to N), which is a number between 5 and 16 depending on how many capabilities are listed under that applied outcome; and c is the score given to a capability by an assessor based on any method used for assessing that capability.
Figure 2. The overall structure of the assessment methodology used for a comprehensive capability metrics assessment tool (CMAT).
Figure 2. The overall structure of the assessment methodology used for a comprehensive capability metrics assessment tool (CMAT).
Sustainability 05 03543 g002
Notes:
  • N, number of applied outcomes under each cluster, is either 2 or 3, depending on the cluster (see Figure 1);
  • M, used to show the number of capabilities under an applied outcome, is an integer between 5 and 16, depending on the applied outcome;
  • For a capability being assessed by an assessor, the average of the scores obtained from different methods will form the overall score of that capability;
  • The average scores of different capabilities under an applied outcome will form the overall score of that applied outcome;
  • The average scores obtained for different applied outcomes under a cluster will form the overall score of that cluster;
  • Each capability is assessed by four assessors, so the same procedure is repeated for each assessor and the averages of all are used;
  • The nature of capability determines what method is used for its assessment.

3. Capability Assessment and Gap Analysis Results

The manufacturer that collaborated with RMIT in this study is an original equipment manufacturer (OEM) and is at the top of a tiered supply chain. Tier one companies supply products directly to the OEM, while tier two companies supply products to tier one companies. Tier three companies supply tier two companies and so on. This study was limited to the assessment of selected tier one and tier two companies. CMAT was used to assess the capabilities relating to sustainable manufacturing within a sample group of tier 1 and 2 apparel and footwear suppliers that participated in this study (Table 2).
Table 2. Apparel and footwear tier 1 and 2 suppliers that participated in the study.
Table 2. Apparel and footwear tier 1 and 2 suppliers that participated in the study.
Supplier NameIndustry TypeShort Description
AFootwear, Tier 1Making sport shoes. The major manufacturing operations include cutting, stitching, gluing, finishing, and packaging.
BFootwear, Tier 1Making sport shoes for major global sporting manufacturers. The main manufacturing operations include cutting, stitching, gluing, finishing, and packaging.
CFootwear, Tier 2Making high-technology plastic shoe components. The major manufacturing processes include design, blending of raw materials, injection molding, and packaging.
DApparel, Tier 1Making sport clothing products with five major divisions including: fabric warehouse; cutting/patterns; sewing; finishing; and sampling.
EApparel, Tier 2Making textile products. The major manufacturing processes include desizing, scouring, washing, mercerizing, bleaching, printing and finishing.
The results of the assessment are compared with the manufacturer’s minimum expectations for different applied outcomes and clusters in order to identify capability gaps and complete a training needs analysis for the sample group of suppliers. Figure 3 shows a summary of the overall results of the capability assessment (applied outcome-based) obtained for the five participating suppliers using CMAT. The details of the gap analysis and required training program have been published by Subic et al. [26].
According to Figure 3, there were some common gaps in capabilities relating to the following applied outcomes:
  • 1.1 Reduce energy use
  • 1.2 Maximize alternative energy resources
  • 2.1 Reduce water use
  • 2.2 Maximize alternative water supply (excluding supplier B)
  • 4.1 Implement and apply waste management hierarchy
  • 4.2 Handle, store, treat, and dispose waste appropriately
  • 5.1 Account for carbon emissions
  • 5.2 Reduce carbon emissions
  • 6.1 Enhance auditing and environmental monitoring performance
  • 6.2 Comply with environmental systems
  • 7.1 Implement industrial clustering and resource pooling
  • 7.3 Identify, develop, and implement business cases for sustainability improvement
The results of this analysis were then used to devise a training program for the suppliers to help them attain the required knowledge and skills in order to meet the environmental targets set by the manufacturer.
Figure 3. The overall results of capability assessment (applied outcome-based), done for five first and second tier footwear and apparel suppliers of a global sporting goods manufacturer, using CMAT (see Figure 1 for details on the applied outcomes 1.1–8.2).
Figure 3. The overall results of capability assessment (applied outcome-based), done for five first and second tier footwear and apparel suppliers of a global sporting goods manufacturer, using CMAT (see Figure 1 for details on the applied outcomes 1.1–8.2).
Sustainability 05 03543 g003
Notes: Considering the nature of the participating suppliers, some applied outcomes may not be applicable for some suppliers (e.g., 7.1 for supplier B). Sustainability 05 03543 i002 The manufacturer’s minimum expectations; Sustainability 05 03543 i003 Applied outcome-based assessment results.

4. Sensitivity Analysis and Assessment Tool Robustness

4.1. Sensitivity Analysis

The following analysis will demonstrate how cluster-based overall assessment results determined by one assessor (A = 1) can be different to that given by another assessor. Following Equation (1), the overall score of a cluster given by an assessor can be obtained using the following equation:
Sustainability 05 03543 i004
Hence, the difference between the overall scores of two assessors can be presented as follows:
Sustainability 05 03543 i005
Sustainability 05 03543 i006 and Sustainability 05 03543 i007 are numbers between 0 and 4. Considering each assessment team as an assessor, the assessment completed by two assessment teams in this study can be assumed as an assessment conducted by two independent representative “assessors”. By reviewing the scores obtained by these two assessors (or two assessment teams) for different capabilities, the maximum differences were mainly about 1 score (excluding some exceptional cases with score differences of about 2 or so). Hence, the maximum effect (when Sustainability 05 03543 i008 for both methods follow the same signs) of such a difference (1 score) for scores given for a capability by two assessors can appear as a difference in the overall score of the cluster as shown in Table 3. As suggested by this equation, the difference in the overall score depends on the number of applied outcomes (N) and the number of capabilities (Mn) under the applied outcome with different given scores:
Sustainability 05 03543 i009
Sensitivity estimations given in Table 3 underline the fact that overestimating or underestimating a capability by one score can lead to a negligible effect on the overall results. That is why the overall scores of different clusters given by two independent assessors show an overall difference of above 1 score in only 7% of cases (grey cells in this table). A further detailed review of the results at applied outcome level also shows that the two assessment teams had agreement on almost the same scores for 60% of the clusters assessed for these suppliers (Figure 4).
Table 3. Sensitivity of the cluster-based results in one score difference for one capability and the cluster-based score differences for five participating suppliers assessed by two independent assessment teams.
Table 3. Sensitivity of the cluster-based results in one score difference for one capability and the cluster-based score differences for five participating suppliers assessed by two independent assessment teams.
Cluster12345678
N 22232232
M1 66161181257
M2 5131294141113
M3 NANANA5NANA12NA
∆C1 (for the 1st applied outcome)0.080.080.030.030.060.040.070.07
∆C2 (for the 2nd applied outcome)0.10.040.040.040.1250.0360.030.04
∆C3 (for the 3rd applied outcome)NANANA0.07NANA0.03NA
Actual ∆C calculated through assessmentSupplier A0.410.221.10.0800.110.010.09
Supplier B0.41.281.060.730.340.6600.04
Supplier C0.050.520.020.120.380.410.570.29
Supplier D0.360.530.010.450.130.250.10.08
Supplier E00.160.050.0900.130.150.35
Note: The overall scores of different clusters given by two independent assessors show an overall difference of above one score in only 7% of cases. These cells are shown in grey color.
Figure 4. Comparison of assessment results obtained by two expert teams from the RMIT University and the manufacturer.
Figure 4. Comparison of assessment results obtained by two expert teams from the RMIT University and the manufacturer.
Sustainability 05 03543 g004

4.2. Supplier Self-Assessment

The self-assessment by participating suppliers was a separate assessment procedure (and not an integral part of the assessment method described in this paper), followed in order to understand the extent to which the suppliers perceive their knowledge against the reference capabilities. These assessments were conducted by the companies’ senior and middle managers, mainly responsible for or involved in maintaining the environmental standards of the participating companies (e.g., ISO 14000 family of standards). A lack of awareness about existing capability gaps can limit companies from improving their practices and moving toward a more sustainable business and meeting the environmental targets set by the manufacturer.
The suppliers were provided with a simple self-assessment questionnaire, including 18 multiple choice questions designed to address the applied outcomes specified in the SMF (Figure 1). The core capabilities and the associated evidence of attainment were indirectly used to design-in the multiple choice answers. The five choices for each question were linked to the rating scores of one to five in order. The suppliers were given access to this assessment questionnaire through a SurveyMonkey online link [39]. Below is an example of self-assessment questions relating to “reduce energy use” applied outcome under the “energy efficiency” cluster:
Question: Which of the following describes best your company’s capability to measure, record, and report energy use and develop options to reduce it?
Answers:
  • We do not measure, record, and report the energy use for the factory (linked to score 1).
  • We measure, record, and report the total energy use for the factory (linked to score 2).
  • We measure, record, and report the total energy use for the factory as well as the energy use of operations/machines (linked to score 3).
  • We measure, record, and report the total energy use of the factory as well as the energy use of operations/machines. The energy use data are used to relate the use in operations to the total use in the factory and identify high use operations (linked to score 4).
  • We measure, record, and report the total energy use for the factory as well as the energy use of operations/machines. These data are used to relate use in operations to the total use in the factory and to identify high use operations. There is at least one demonstrable example of energy efficiency improvement within the factory implemented through following such an approach (linked to score 5).
The self-assessment results are presented in Figure 5. As expected, these results did not generally show sufficient reliability in identifying the suppliers’ capability gaps. “Carbon emissions” was the only cluster showing relatively-close agreement between the suppliers’ self-assessment and the assessment done based on CMAT by the expert teams from RMIT and the manufacturer (except Supplier B). Capabilities in other clusters were generally overestimated according to the suppliers’ self-assessment results. Such discrepancies between assessment results were not unexpected as an existing lack of suppliers’ knowledge in a particular cluster may not place well that supplier in critically assessing their capabilities across that cluster. Note that almost the same ranking orders for the suppliers in terms of their capabilities are suggested by the RMIT/manufacturer assessment teams, and by the suppliers’ self-assessment results (Figure 5).
Figure 5. The same order in supplier rankings based on their sustainability performance is suggested by the RMIT/manufacturer assessment teams using CMAT, and by the suppliers’ self-assessment using a simple short questionnaire. (a) RMIT/Manufacturer assessment results; (b) Supplier self-assessment results; (c) The average of clusters’ scores assessed by RMIT/Manufacturer team and self-assessed by the A–E participating suppliers.
Figure 5. The same order in supplier rankings based on their sustainability performance is suggested by the RMIT/manufacturer assessment teams using CMAT, and by the suppliers’ self-assessment using a simple short questionnaire. (a) RMIT/Manufacturer assessment results; (b) Supplier self-assessment results; (c) The average of clusters’ scores assessed by RMIT/Manufacturer team and self-assessed by the A–E participating suppliers.
Sustainability 05 03543 g005

4.3. Results Discussion

The inherent interconnections between the assessment results of different clusters can be used to evaluate the validity of the results and comment on the reliability of the assessment tool. Some of the main links between the clusters are discussed below.

4.3.1. Connection between Three Groups of Clusters

It is expected that a lack of capability in “improving environmental management practices” (clusters 6, 7 and 8 in Figure 1) is reflected in the lack of capability in using energy, water, and material resources efficiently (clusters 1, 2 and 3) and in achieving emissions reduction. One should be skeptical about the reliability of the assessment tool if the results show that a supplier with poor performance in capabilities relating to “improved management practices” has obtained a high score in that relating to resource efficiency and emission reduction. This also agrees with the generally-accepted definition of manufacturing environmental management as management practices to minimize the negative environmental impacts of the manufacturing activities [25]. In searching for such a connection, the average scores for clusters under resource efficiency, emissions reduction, and improved management practices (three main cluster categories as shown in Figure 1) were calculated for all five suppliers that participated in this study (Figure 6). Although different capabilities were assessed under each of these three categories of management, the average scores of the categories follow almost the same ranking order for the five participating suppliers. In other words, the supplier with the higher average score in the “improved management practices” capability cluster also obtained higher average scores in the other two cluster categories. Interestingly, for each individual supplier, the average scores for these three groups of clusters are all approximately the same with a maximum difference of 0.6 only (for Supplier A) and almost the same scores for Supplier C (2.42, 2.42, and 2.46). A similar link was also identified by Wong et al. [4] in studying 122 manufacturing firms investigating the role of environmental management capabilities in achieving green operation (GO) at the manufacturing level; Lo et al. [17] also studied the role for proper environmental management systems (EMSs) in controlling the environmental performance of manufacturing firms, particularly in developing countries where the implementation of such EMSs faces challenges such as lack of appropriate infrastructure, efficient policies and effective environmental regulations, and financial resources [40].
Figure 6. Average scores in resource efficiency, emissions reduction, and improved management practices cluster categories for the five participating suppliers.
Figure 6. Average scores in resource efficiency, emissions reduction, and improved management practices cluster categories for the five participating suppliers.
Sustainability 05 03543 g006

4.3.2. Integration of Energy Efficiency and Carbon Emission Clusters

Suggested energy management hierarchies for sustainability purposes have usually integrated carbon and energy-related capability, e.g., [41] also used by Smith and Ball [38], as energy can often serve as a proxy for carbon emissions, particularly for fossil-based energy systems. In fact, although some capabilities relating to “carbon emissions” and “energy efficiency” are connected, good capabilities in one cluster do not necessarily mean good capabilities in other clusters. In some instances, carbon emissions can be reduced without any reduction in energy consumption: for example, switching to photovoltaic electricity does not reduce energy consumption but can reduce carbon emissions. Furthermore, some manufacturers established targets based on separate energy and carbon emission indicators, e.g., [18]. For this reason, the integration of capabilities relating to carbon emissions and energy is considered not appropriate.
As shown in Table 4, the majority of capabilities in the carbon emissions cluster relate to those required to undertake different levels of carbon accounting which are different to those outlined under the energy efficiency cluster. However, as discussed, the existing links are reflected in the close scores for these two clusters as shown in Figure 7 for the five participating suppliers. Following Figure 7, the biggest gap between the two clusters (0.4) was for supplier B; interestingly, this supplier utilized onsite electricity generation using diesel generators. Hence, they could potentially reduce their greenhouse gas emissions while keeping their electricity consumption constant.
Table 4. Capabilities included under the “energy efficiency” and “carbon emissions” clusters.
Table 4. Capabilities included under the “energy efficiency” and “carbon emissions” clusters.
ClusterEnergy efficiencyCarbon Emission
CapabilitiesCapabilities Under
Reduce Energy Use
Applied Outcome
Measure, record and report energy used in operations for factoryCapabilities Under
Account for Carbon Emissions
Applied Outcome
State the three main greenhouse gases for energy processes
Compare energy use records for operations to previous records and industry best practiceRecognize other greenhouse gas emissions, sources (e.g., some refrigerant gases) and characterisation factors
Relate energy used in operations to total energy consumption for factoryList or name global warming factors (CO2-equivalent factors) for main greenhouse gases
Use ranking and identified energy losses to determine processes which can be improvedor produce a document which lists emission factors for different energy types (e.g., fuel oil, electricity)
Document a flow chart showing energy flows for an operation in the factory (energy balance)Calculate greenhouse gas (GHG) emissions from energy use and emission factors
Apply continuous improvementReport on GHG emissions for the factory
Capabilities Under
Maximize Alternative Energy Resources
Applied Outcome
Determine alternative energy sources (e.g., heat from gas vs. heat from electricity, solar)Compare GHG emissions to previous records
Implement alternative energy sourcesDifferentiate between Scope 1, 2 and 3 emissions
Operate alternative energy resource technology (s)Capabilities Under
Reduce Carbon Emissions
Applied Outcome
Use GHG emissions and energy data to find operations contributing to GHG impacts
Maintain alternative energy resource technology (s)Determine options to reduce GHG emissions
Recommend changes to reduce GHG emissions
Apply continuous improvementImplement changes to reduce GHG emissions
Figure 7. Close scores for energy efficiency and carbon emissions clusters obtained by all of the five suppliers that participated in the study.
Figure 7. Close scores for energy efficiency and carbon emissions clusters obtained by all of the five suppliers that participated in the study.
Sustainability 05 03543 g007

5. Conclusion

The paper presented a comprehensive Sustainable Manufacturing Framework (SMF) that was developed in collaboration with a global sporting goods manufacturer. This tool was used to identify the key requirements for building and/or enhancing capabilities across the manufacturing supply chain in Asia in order to promote and secure the uptake of environmentally sustainable manufacturing practices. The framework was designed to particularly target the key elements of resource management, emissions management and environmental management to help the manufacturer reduce their environmental footprint and meet their strategic measurable environmental targets. This framework was used as a platform to develop the Capability Metrics Assessment Tool (CMAT). The CMAT was used by authors to assess a select group of five tier one and tier two sports apparel and footwear suppliers. The assessment results obtained in this research identified distinctive capability gaps, characterized by the lack of capability to:
  • Identify the resource/emissions intensive processes (or to conduct a “Hot-Spot” Analysis of their manufacturing operations);
  • Identify the reasons for this intensity (or to conduct a “Root-Cause” Analysis);
  • Evaluate the costs and benefits associated with environmental improvement options (or to conduct a “Full-Cost” analysis);
  • Decide on a sustainable option using multi-criteria assessment (or to conduct a “Multi-Criteria Decision-Making” process).
Five suppliers were assessed by two expert assessment teams from the university and the manufacturer using four independent assessment methods; the differences between the obtained scores were mainly less than 1. Such small differences between the capability assessment scores led to the conclusion that 93% cluster-based score agreements between the two assessment teams were achieved if the agreement is defined as a score difference of 1 or less. In only three cases (highlighted in grey in Table 3) the overall scores of the clusters showed differences of slightly above 1. By reviewing the cluster-based scores obtained by the two assessment teams, it is apparent that there is an agreement on almost the same scores in 60% of the clusters assessed for the same group of suppliers (Figure 4). There are also some important connections between the scores of different clusters which confirmed the robustness and the reliability of the assessment tool and the methodology used. The assessment results for all the participating companies illustrated that those with poor performance in environmental management practices could not use their energy, water, and material resources efficiently in order to reduce their environmental footprint (e.g., waste, chemicals, and carbon emissions) effectively (Figure 6). Furthermore, the results clearly demonstrated links between the “Carbon Emissions” and “Energy Efficiency” capability clusters. Companies with low scores in the latter received low scores in capabilities relating to their ability to reduce their carbon emissions (Figure 7). Finally, the suppliers completed a self-assessment against the same set of capabilities. Whilst the self-assessment results did not agree with those determined by the expert teams from the university and the manufacturer both set of assessments resulted in the same ranking order of the suppliers (Figure 5).
In conclusion, the key focus in this paper was on the analysis of the obtained assessment results in order to determine the robustness of the developed CMAT tool. This was important in order to demonstrate how reliably the tool can be applied to assess a greater number of suppliers across the region and globally.

Acknowledgments

The authors wish to acknowledge the invaluable support of the Adidas Group without which this research would not be possible.

Conflict of Interest

The authors declare no conflict of interest.

References

  1. Jayal, A.D.; Badurdeen, F.; Dillon, O.W., Jr.; Jawahir, I.S. Sustainable manufacturing: Modeling and optimization challenges at the product, process and system levels. CIRP—JMST 2010, 2, 144–152. [Google Scholar]
  2. Choi, T.-M.; Lo, C.K.Y.; Wong, C.W.Y.; Yee, R.W.Y. Green manufacturing and distribution in the fashion and apparel industries. Int. J. Prod. Econ. 2012, 135, 531. [Google Scholar] [CrossRef]
  3. MSA sustainable manufacturing for sustainability. Available online: http://www.mskills.com.au/ (accessed on 4 July 2013).
  4. Wong, C.W.Y.; Lai, K.-H.; Shang, K.-C.; Lu, C.-S.; Leung, T.K.P. Green operations and the moderating role of environmental management capability of suppliers on manufacturing firm performance. Int. J. Prod. Econ. 2012, 140, 283–294. [Google Scholar] [CrossRef]
  5. Caniato, F.; Caridi, M.; Crippa, L.; Moretto, A. Environmental sustainability in fashion supply chains: An exploratory case based research. Int. J. Prod. Econ. 2012, 135, 659–670. [Google Scholar] [CrossRef]
  6. Hofmann, K.H.; Theyel, G.; Wood, C.H. Identifying firm capabilities as drivers of environmental management and sustainability practices—evidence from small and medium-sized manufacturers. Bus. Strat. Environ. 2012, 21, 530–545. [Google Scholar] [CrossRef]
  7. Simpson, D. Knowledge resources as a mediator of the relationship between recycling pressures and environmental performance. J. Clean. Prod. 2012, 22, 32–41. [Google Scholar] [CrossRef]
  8. Kaebernick, H.; Kara, S.; Sun, M. Sustainable product development and manufacturing by considering environmental requirements. Rob. Cim.—Int. Manuf. 2003, 19, 461–468. [Google Scholar] [CrossRef]
  9. Zhu, Q.H.; Sarkis, J.; Lai, K.-H. Confirmation of a measurement model for green supply chain management practices implementation. Int. J. Prod. Econ. 2008, 111, 261–273. [Google Scholar] [CrossRef]
  10. Schoenherr, T. The role of environmental management in sustainable business development: A multi-country investigation. Int. J. Prod. Econ. 2012, 140, 116–128. [Google Scholar] [CrossRef]
  11. Rao, P. The greening of suppliers—in the south east asian context. J. Clean. Prod. 2005, 13, 935–945. [Google Scholar] [CrossRef]
  12. Yang, M.G.; Hong, P.; Modi, S.B. Impact of lean manufacturing and environmental management on business performance: An empirical study of manufacturing firms. Int. J. Prod. Econ. 2011, 129, 251–261. [Google Scholar] [CrossRef]
  13. Parast, M.M.; Adams, S.G. Corporate social responsibility, benchmarking, and organizational performance in the petroleum industry: A quality management perspective. Int. J. Prod. Econ. 2012, 139, 447–458. [Google Scholar] [CrossRef]
  14. Gunasekaran, A.; Spalanzani, A. Sustainability of manufacturing and services: Investigations for research and applications. Int. J. Prod. Econ. 2012, 140, 35–47. [Google Scholar] [CrossRef]
  15. Rosen, M.A.; Kishawy, H.A. Sustainable manufacturing and design: Concepts, practices and needs. Sustainability 2012, 4, 154–174. [Google Scholar] [CrossRef]
  16. Bi, Z. Revisiting system paradigms from the viewpoint of manufacturing sustainability. Sustainability 2011, 3, 1323–1340. [Google Scholar] [CrossRef]
  17. Lo, C.K.Y.; Yeung, A.C.L.; Cheng, T.C.E. The impact of environmental management systems on financial performance in fashion and textiles industries. Int. J. Prod. Econ. 2012, 135, 561–567. [Google Scholar] [CrossRef]
  18. The adidas group environmental strategy 2015. Available online: http://www.adidas-group.com/en/sustainability/assets/environmental_statements/adidasGroup_EnvironmentalStrategy.pdf (accessed on 4 July 2013).
  19. Sharma, S.; Vredenburg, H. Proactive corporate environmental strategy and the development of competitively valuable organizational capabilities. Strategic Manage. J. 1998, 19, 729–753. [Google Scholar] [CrossRef]
  20. De Bakker, F.; Nijhof, A. Responsible chain management: A capability assessment framework. Bus. Strat. Environ. 2002, 11, 63–75. [Google Scholar] [CrossRef]
  21. Kinderytė, L. Analysis and comparison of methodologies for corporate sustainability assessment. EREM 2008, 4, 66–75. [Google Scholar]
  22. Sustainability—competency, opportunity, reporting & evaluation. Available online: http://www.sustainability-core.com/ (accessed on 4 July 2013).
  23. Seuring, S.; Müller, M. From a literature review to a conceptual framework for sustainable supply chain management. J. Clean. Prod. 2008, 16, 1699–1710. [Google Scholar] [CrossRef]
  24. Yuan, C.; Zhai, Q.; Dornfeld, D. A three dimensional system approach for environmentally sustainable manufacturing. CIRP Ann.—Manuf. Techn. 2012, 61, 39–42. [Google Scholar] [CrossRef]
  25. Humphreys, P.; McIvor, R.; Chan, F. Using case-based reasoning to evaluate supplier environmental management performance. Expert Syst. Appl. 2003, 25, 141–153. [Google Scholar] [CrossRef]
  26. Subic, A.; Shabani, B.; Hedayati, M.; Crossin, E. Capability framework for sustainable manufacturing of sports apparel and footwear. Sustainability 2012, 4, 2127–2145. [Google Scholar] [CrossRef]
  27. Judge, W.Q.; Elenkov, D. Organizational capacity for change and environmental performance: An empirical assessment of bulgarian firms. J. Bus. Res. 2005, 58, 893–901. [Google Scholar] [CrossRef]
  28. Reuter, C.; Foerstl, K.A.I.; Hartmann, E.V.I.; Blome, C. Sustainable global supplier management: The role of dynamic capabilities in achieving competitive advantage. JSCM 2010, 46, 45–63. [Google Scholar]
  29. Adidas sustainability. Available online: http://www.adidas-group.com/en/sustainability/welcome.aspx/ (accessed on 4 July 2013).
  30. NTIS Training codes: Bcpps5004a; bsbpur501a; bsbmgt608b; cpccbc4019a; icpsu261c; icpsu323b; ictsus7235a; rtc1701a; srxres004b; srxres005b; tdtu601a; and thcghs04b. Available online: http://training.gov.au/ (accessed on 4 July 2013).
  31. DEEWR Training packages codes: Ict10; tli07; prm04; nwp07; psp04; and pma08. Available online: http://www.deewr.gov.au/Pages/default.aspx/ (accessed on 4 July 2013).
  32. International Organization for Standardization (ISO), Guidelines for Quality and/or Environmental Management Systems Auditing; ISO: Geneva, Switzerland, 2002; Vol. ISO 19011:2002.
  33. International Organization for Standardization (ISO), Environmental Management Systems—Requirements with Guidance for Use; ISO: Geneva, Switzerland, 2004; Vol. ISO 14001:2004.
  34. Adidas strategy 2015, impacts along the value chain. Available online: http://www.adidas-group.com/en/SER2010/environment/strategy-2015/impacts-along-the-value-chain/Default.aspx/ (accessed on 4 July 2013).
  35. Adidas sustainability report 2010, in the real world performance counts. Available online: http://www.adidas-group.com/en/SER2010/_assets/downloads/adidasSR2010_full.pdf (accessed on 4 July 2013).
  36. MSA sustainable manufacturing unpacking the issues. Available online: www.mskills.com.au/ (accessed on 4 July 2013).
  37. Herva, M.; Franco, A.; Carrasco, E.F.; Roca, E. Review of corporate environmental indicators. J. Clean. Prod. 2011, 19, 1687–1699. [Google Scholar] [CrossRef] [Green Version]
  38. Smith, L.; Ball, P. Steps towards sustainable manufacturing through modelling material, energy and waste flows. Int. J. Prod. Econ. 2012, 140, 227–238. [Google Scholar] [CrossRef] [Green Version]
  39. SurveyMonkey. Available online: http://www.surveymonkey.com/ (accessed on 4 July 2013).
  40. Massoud, M.A.; Fayad, R.; Kamleh, R.; El-Fadel, M. Environmental management system (ISO 14001) certification in developing countries: Challenges and implementation strategies. J. Environ. Sci. Technol. 2010, 44, 1884–1887. [Google Scholar] [CrossRef]
  41. IET energy principles. Available online: http://www.theiet.org/factfiles/energy/energy-prin-page.cfm/ (accessed on 4 July 2013).

Share and Cite

MDPI and ACS Style

Subic, A.; Shabani, B.; Hedayati, M.; Crossin, E. Performance Analysis of the Capability Assessment Tool for Sustainable Manufacturing. Sustainability 2013, 5, 3543-3561. https://doi.org/10.3390/su5083543

AMA Style

Subic A, Shabani B, Hedayati M, Crossin E. Performance Analysis of the Capability Assessment Tool for Sustainable Manufacturing. Sustainability. 2013; 5(8):3543-3561. https://doi.org/10.3390/su5083543

Chicago/Turabian Style

Subic, Aleksandar, Bahman Shabani, Mehdi Hedayati, and Enda Crossin. 2013. "Performance Analysis of the Capability Assessment Tool for Sustainable Manufacturing" Sustainability 5, no. 8: 3543-3561. https://doi.org/10.3390/su5083543

Article Metrics

Back to TopTop