Development of Manufacturing Sustainability Assessment Using Systems Thinking
1
Department of Technology and Management, Gjøvik University College, 2815 Gjøvik, Norway
2
Department of Engineering Design and Materials, Norwegian University of Science and Technology, 7491 Trondheim, Norway
*
Author to whom correspondence should be addressed.
Sustainability 2016, 8(1), 5; https://doi.org/10.3390/su8010005
Submission received: 30 October 2015
/
Revised: 19 December 2015
/
Accepted: 20 December 2015
/
Published: 22 December 2015
(This article belongs to the Section Economic and Business Aspects of Sustainability)
Abstract
:The existing body of knowledge in sustainability assessment of manufacturing indicates that although extensive research is going on, significant shortcomings remain unsolved. In this paper, the specific needs of a new sustainability assessment system are discussed. Systems thinking is suggested as an alternative to the reductionist approach which is commonly applied to sustainability assessment. Although previous research has recognized the potential of systems thinking applied to sustainability assessment, few practical examples have been demonstrated. Therefore, this article focuses on a practical application of systems thinking to the development of a sustainability assessment system of a manufacturing organization. A framework for development is proposed employing systems thinking. It is suggested that systems thinking reveals several aspects usually not addressed by the reductionist approaches. It is demonstrated that a combination of tools like analysis of multiple viewpoints, conceptagon, seven samurai, and model-based systems engineering can enhance a development of a suitable assessment systems.
1. Introduction
Sustainability assessment is an essential means to promote sustainable development. An extensive research has been carried out on sustainability assessment discourses and pluralism of sustainable development and sustainability assessment, e.g., [1,2,3,4]. The plural nature of sustainability assessment has led to a variety of sustainability assessment approaches and tools, see e.g., [5,6,7].
Despite the growing interest in application of sustainability assessment in the manufacturing and a great number of developed assessment tools (e.g., [8,9,10,11,12]), only few of them have been used by manufacturing companies. One of the reasons is that sustainability assessment tools may appear too theoretical and abstract [9,13], or too technical and complicated for manufacturing companies [14,15,16]. Researchers attempt to create the ideal assessment tool, whereas manufacturers do not know from where to start their journey toward sustainability, causing a gap between the two communities. In addition, some of the existing frameworks for indicators are aimed at external reporting, rather than providing valuable information for internal decision makers [17]. This focus only on external reporting may create a barrier to use the assessment since the output value is not in balance with the resources required to use the tool, especially if the resources are scarce. As a result, companies are not willing to spend the amount of resources required for employing tools such as life cycle assessment, environmental impact assessment, life cycle costing, process energy analysis, social life cycle assessment, and cost-benefit analysis [18,19,20,21]. Most of these tools are resource intensive and only address one or two sustainability aspects. Thus, manufacturing organizations may find it difficult to choose the appropriate tool or combination of tools. Especially if the outcome of the assessment is inconclusive when it comes to establishing the “optimal” assessment method for manufacturing companies. On one side, researchers aim to ensure fulfillment of all scientific requirements, e.g., intergenerational equity, and consideration of the normative nature of sustainability [19], on the other side, manufacturing companies have needs and expectations that might differ from the scientific ones. Thus, sustainability assessments must provide a balance between different types of requirements.
In addition, seeing manufacturing as a “system of systems” brings in additional sources of complexity which the sustainability assessment tool has to deal with. Unfortunately, most of the existing assessment tools are incapable of capturing the complexity of manufacturing. Moldavska and Welo [22] have analyzed the applicability of existing sustainability assessment tools in manufacturing and concluded that a tool—which identifies specific problems in the company based on reliable information with limited resources while addressing three pillars of sustainability with meaningful normalization, weighting, and aggregation procedure—is still lacking.
Due to problems that manufacturing organizations face while trying to adapt an assessment tool to their specific business context, we emphasize a need to focus on a “sustainability assessment system”. Often developers of sustainability assessment tools focus on the tool by itself and not on its application capabilities. Thus, applicability of an assessment tool can be increased by considering the environment in which the tool will be used. Here, “sustainability assessment system” is defined as a system that addresses a manufacturing organization and provides valuable input to decision-makers regarding sustainable development (Figure 1). The term “sustainability assessment system” is used to define any analytical technique, e.g., guideline, algorithm or software, and people involved in data collection and communication with decision makers. An assessment system differs from an assessment process—the broad steps taken to undertake an assessment—and an assessment tool—analytical techniques employed within the process [1].
Figure 1.
Sustainability assessment—a system that assess a system.
One of the more recent researches into sustainability assessment points out that traditional sustainability assessments based upon linear cause-and-effect thinking are inadequate [23]. Thus, an alternative approach should be identified. A systems approach or systemic mindset has been propounded to address a complex sustainability concept instead of a reductionist one, e.g., [2,23]. However, according to Grace and Pope [23], beyond the largely conceptual contributions to recognize the potential of a system approach to sustainability assessment, relatively little has been done to incorporate systems thinking into sustainability assessment—and even fewer practical examples have been demonstrated. Grace and Pope argue that the full potential of systems thinking is constrained by a lack of skills within the assessment community in employing systems tools, as well as by ambiguity about the concept of sustainability and the goals of the assessment. In addition, Bell and Morse [2] emphasize that such an alternative approach “becomes terribly difficult”.
The primary driver for this research is the need to overcome challenges and shortcomings of the current sustainability assessments in manufacturing. The secondary driver is the conclusion of one of the latest research conducted by Grace and Pope [23]. The authors conclude that although application of system-based methods to sustainability assessment is in its infancy, the employment of these methods is one of the opportunities in future direction for sustainability assessment. Thus, we aim at exploring this opportunity as a means to overcome shortcomings of current sustainability assessments. The objective of this article is to show how developers of sustainability assessments may incorporate ideas of systems thinking into a sustainability assessment of a manufacturing organization. To achieve this objective, we will briefly review the nature of sustainability assessment first (Section 2). Second, we will emphasize the need for a new sustainability assessment by reviewing shortcomings of existing sustainability assessment tools (Section 3). Third, we review the underlying ideas of systems thinking and discuss how sustainability assessment may benefit from systems thinking (Section 4). Then we propose a framework for development of sustainability assessment and demonstrate how the use of each tool (multiple views analysis, conceptagon, seven samurai, and model-based systems engineering) enables incorporation of systems thinking ideas into sustainability assessment (Section 5).
2. Sustainability Assessment
Although the terms “sustainable development” and “sustainability assessment” are widely used, each is interpreted in a variety of ways. The lack of consensus on what sustainable development is has been addressed by researchers. For example, Pope et al. [1] outlined four discourses: (1) the pragmatic integration of development and environmental goals; (2) the idea of limitations on human activities; (3) a process of directed change or transition; and (4) enhancing resilience and justice. Similarly, Hopwood et al. [3] have mapped different views within the sustainable development debate, e.g., status quo, reform and transformation approaches, strong and weak sustainability. Bell and Morse [2] have studied a diversity of viewpoints on the meaning of sustainability and emphasized the need to accommodate this diversity rather than seek to remove it. However, diverse interpretations of sustainable development concept determine the pluralism of sustainability assessment. Bond et al. [4] stress that the ecological, social, political and cultural pluralism provides a plural context for sustainability assessment. Moreover, Cashmore and Kørnøv [24] outline two dimensions of plurality; plurality of theoretical perspectives on sustainability assessment and plurality of stakeholders with multiple perspectives. Due to the plurality of sustainability assessment, “no single, definitive and globally agreed sustainability assessment process is likely to emerge beyond some basic steps” [25]. However, according to Marsden et al. [26], an incomplete definition of sustainability assessments leads to inability of current methods to capture the full range of concerns. Diversity of viewpoints on what sustainability assessment is and how to perform it have led to a vast number of sustainability assessments. The term “sustainability assessment” is used as an umbrella for different procedures, practices, processes, methodologies, methods, frameworks, and tools that focus on measuring or promoting sustainability at different levels, e.g., country, city, and organization. Existing definitions of sustainability assessment include:
- “the process that directs decision-making towards sustainability” [27];
- “a process that aims to integrate sustainability issues into decision-making by identifying sustainability impacts, but also by fostering sustainability objectives to be achieved” [28];
- “an important tool to aid in the shift towards sustainability… and a process by which the implications of an initiative on sustainability are evaluated, where the initiative can be a proposed or existing policy, plan, programme, project, piece of legislation, or a current practice or activity” [6];
- “a tool that can help decision-makers and policy-makers decide what actions they should take and should not take in an attempt to make society more sustainable” [29];
- “the tool that can help decision and policymakers select which actions are to be taken to make society more sustainable” [30].
A variety of sustainability assessment approaches exists and researchers have developed different categorization frameworks to systematize these approaches (Table 1). In addition, a framework for comparing sustainability assessment processes and evaluating the effectiveness of the assessment have been developed by Bond et al. [4]. The framework includes six criteria—procedural effectiveness, substantive effectiveness, transactive effectiveness, pluralism, knowledge and learning,—which reflect a variety of sustainability assessment functions for decision-making.
| Paper | Categorization Dimensions | Approaches to Sustainability Assessment |
|---|---|---|
| Ness et al. [5] |
|
|
| Pope et al. [6] |
| |
| Pope et al. [1] based on [6] |
| |
| Videira et al. [7] based on [6] |
| |
| Wiek and Binder [31] |
| |
| Pope et al. [1] |
| |
| Huge et al. [32] |
| |
| Hacking and Guthrie [33] |
|
The purpose of sustainability assessment has been widely discussed and different opinions exist. Bell and Morse have raised a concern, that by attempts to quantify sustainability “we end up measuring things that can be measured and not things that should be measured” [2]. Moreover, Grace and Pope [23] point out that a sustainability assessment should address the goal of sustainability, e.g., enduring human wellbeing. Waas et al. [34], who have extended the focus of sustainability assessment from a support to decision makers to a learning and participation support tool, outlined four purposes of sustainability assessment:
- Information generation for decision-making;
- Operationalization and forum for participation, debate and deliberation;
- Social learning;
- Structuring complexity.
3. The Need for a New Sustainability Assessment Tool
In this section, challenges and shortcomings of sustainability assessment tools are reviewed as well as the need for a new sustainability assessment tool and a potential for improvements are illustrated.
According to Pope et al. [6], although a sustainability assessment is important to aid a shift in practices, very few examples of effective assessment processes can be found. According to Grace and Pope [23], a traditional sustainability assessment based upon linear cause-and-effect thinking is inadequate. The authors state that an ideal form of a sustainability assessment would be a comprehensive assessment that spans over all pillars of sustainability, fully connected in terms of the covered themes and techniques used, and forward-looking. Such tool does not exist at present, though. The triple bottom line representation of sustainability with its list of indicators brings the following challenges to the table: how to integrate the separate categories and pillars, and how to manage trade-offs within these processes? These two challenges are the key research drivers in finding an alternative approach to sustainability assessment. The knowledge and shortcomings associated with sustainability assessments have been identified through a literature review and are presented below. The identified shortcomings are divided into two categories: methodology behind a tool, and implementation of a tool.
3.1. Shortcomings: Methodology behind a Tool
Due to the pluralism inherent in the sustainable development concept, different definitions of sustainable manufacturing has been proposed, e.g., [13]. Some authors stress the need for establishing a clear understanding of sustainable manufacturing [35]. Others argue that it is necessary to move from trying to identify sustainable manufacturing toward developing tools for measuring achievements [36]. Due to little consensus as to how to define and measure sustainable manufacturing [36], every assessment tool represents an individual view and approach. A number and a variety of indicators also indicate that no consensus exist on how to assess sustainable manufacturing [14,16,35,36,37,38]. The variety of assessment tools is driven by different approaches to sustainability assessment—sustainability criteria, meaning that each assessment presents different sustainability performance [39]. Moreover, limited consensus exists on a reasonable taxonomy of sustainability metrics [40]. As a result, a company may achieve high sustainability performance according to one assessment and low performance according to another. Thus, contradictory strategies for how to improve sustainability performance can be established due to the compositions and interpretations of the indicators [35].
There is an ongoing discussion among researchers regarding selection of indicators and metrics. Some authors argue that it is not possible to rely on ready-made sets of indicators due to severe gaps for some areas, such as land use, and comparison between renewable and non-renewable resources [41]. Ameta et al. [42] concur that the definitions of key sustainability metrics are inconsistent and business-specific. Others state that it is better to focus on a small widely applicable set of sustainability metrics rather than to include a full set of industry-specific metrics [43]. Veleva and Ellenbecker [37] argue that “while the number of sustainability indicators in the literature is growing, none of them advances the understanding of corporate sustainability”. Taisch et al. [39] support their viewpoint by stating that a set of comprehensive indicators for sustainability assessment is still lacking. According to Ameta et al. [42], metrics that would be uniformly defined and globally harmonized are clearly missing. There is no standard approach for developing indicator sets as every new tool defines sets that overlap with existing ones. When taking into account a number of scattered indicators and metrics for different organizational levels and different sustainability aspects, it is essential to employ a set of interconnected indicators appropriate for analyzing performance of a complex socio-technical system. However, a holistic approach to sustainability and visibility of relationships between metrics/indicators are still missing [44]. An appropriate choice of indicators is one of the essential parts of a sustainability assessment. Optimization of one component of the system due to the set of indicators may lead to sub-optimization of the system [45]. Some researchers stress that many indicator sets focus on a too limited scope of sustainability and fail to capture the entire system perspective, see e.g., [46]. While others emphasize a plural nature of sustainability assessment see e.g., [1,4], meaning that a fixed set of indicators cannot address this pluralism. Although it is clearly important to address this interrelated complexity of sustainability assessments, many existing solutions focus on separate subsystems (production, product development, work-cell, etc.) or problems (energy efficiency, CO2 reduction, recycling, etc.). Sub-optimization within the organization can also be caused by the choice of assessment level (product, process, or activity). If the assessment tool focuses explicitly on production, product development, or logistics, the rest of the organization may be affected by decisions made without fully considering the implications. In other words, many sustainability assessment tools often miss a holistic perspective. In addition, few, if any, assessment tools have a holistic view on couplings and interrelationships between activities, hence failing to assess the company as a system [22].
A company commonly selects appropriate sustainability indicators based on strategy or type of business. Thus, different companies aim to achieve different values and sustainability performance; for example, “even when a company use experts to select the right set of indicators, this step is subjective as selection of the right set of indicators depends on many factors”, according to Joung et al. [35]. Another reason why different companies try to achieve different sustainability performance is specific benchmarking values. For instance, the benchmarking value of an indicator can be based on past performance of the company, a particular standard, the amount of reduction required in a specific period [35], a defined target, or similar manufacturing systems [47].
There are several other shortcomings associated with sustainability assessment tools. First, it is difficult “to identify current measurable indicators that point to sustainability” since sustainability is associated with the future while indicators measure the present [48]. Second, “sustainability indicators in fact are calculating by gauging, comparing, correlating these quantities during a specified period of time, blind to the dynamics of the manufacturing processes in that period of time” [47]. Moreover, it is not an easy task to compare and correlate different indicators. Paju et al. [14] argue that one of the challenges is that different types of indicators use different reference units (work cycle, yearly production volume, days, product, etc.). In addition, researchers indicate that three spheres of sustainability have not received equal attention during sustainability assessment, and how to measure the social dimension remains a major problem in recent research [36]. The social dimension is usually the least covered sustainability pillar, although some efforts have been done to incorporate the social dimension into assessment [44,49]. In contrast, other researchers emphasize the need to consider an integrated sustainability assessment and do not separate sustainability into three pillars, see e.g., [23]. Integrated sustainability assessment have been discussed by Gibson [50], who stressed the need to consider sustainability as an integrative concept instead of addressing three pillars separately. Gibson argued that this might be achieved by designing sustainability assessment “more aggressively as an integrative process”.
3.2. Shortcomings: Implementation of a Tool
Sustainability assessment tools are usually resource intensive, and the more comprehensive assessment tools require significant time for data collection. Some researchers relate this to the existence of a gap between theory and practices [39]. In addition, many existing sustainability assessment methods require multi-disciplinary, in-depth knowledge for implementation (AHP, fuzzy logic, value stream mapping, discrete event system, etc.), which may be lacking in many companies.
Manufacturing companies are facing the challenge of identifying appropriate assessment tools due to their variety and difference between the indicators. The existence of many indicator sets has created confusion when a manufacturing company attempts to select a suitable tool [35]. In addition, a variety of issues can affect the applicability of assessment tools. Moldavska and Welo [22] provide a list of challenges related to applicability of sustainability assessment tools: (1) assessment tools may provide a too general or too site-specific approach; (2) may be too theoretical and complicated; (3) point out problems at too general level; (4) lack reliability of the result (e.g., due to subjectivity during an aggregation and weighting procedures); (5) lack in addressing case company’s context; and (6) focus mainly on external reporting.
An assessment tool usually has the ambition to evaluate current sustainability performance, allowing a company to make a decision regarding future possible improvements. However, the literature review indicates that the link between sustainability assessment and sustainability practices is still lacking [22]. The main reason is that assessments often focus on external reporting. Moreover, the same sustainability practice may not fit two companies with the same problem. There is, however, recent and ongoing research work in this field. Fischer et al. [51] presented a method for selecting solutions for improving energy efficiency based on the assessment of the current situation in the company. Despeisse et al. [52] link an assessment of manufacturing process resource flows with manufacturing tactics. This approach should guide manufacturers to contribute to sustainability through the factory modeling, resource flow analysis, and improvement opportunities identification.
The common denominator for all the problems presented above is the complexity of manufacturing organizations and sustainability concept. What to assess, i.e., which part of the company, and how to assess, i.e., criteria, are the main questions that must be carefully addressed by a future assessment tool. Any manufacturing company is a dynamic “system of systems” with a high degree of complexity. Additional sources of complexity are different assessment levels (product, process, company, etc.), different sustainability dimensions (social, economic, and environmental), different perspectives, and different time references [47].
3.3. Lessons Learned
Although many shortcomings exist, this is what drives researchers to explore better solutions. Research made during the last years has yielded good progress. The following review is by no means complete and gives only a brief overview of the present progress.
A conceptual manufacturing ecosystem model, proposed by Despeisse et al. [53], allows a better understanding of manufacturing complexity and interactions between manufacturing operations, facilities, and buildings. Although the model is limited to factory level and environmental issues, this particular approach allows identification of improvements through a structured sustainability assessment. Chen et al. [11] proposed an approach to the development of a sustainability assessment model that can provide decision support for improvements. The model is built upon three modules—sustainability framework, factory model, and indicator framework—which are connected through connection matrices. Lanz et al. [44], who emphasized that relationships between metrics/indicators are still missing, developed a set of interconnected indicators for analyzing the performance of a socio-technical system. The authors point out that a visualization of cause-effect relationships between qualitative and quantitative indicators can facilitate the development process in the company. Fantini et al. [47] presented a framework for key performance indicators built on only a few main indicators that allow capturing the whole eco-factory performance. This work contributes to a possible performance management system that can enable a systematic assessment of manufacturing behavior from different viewpoints. This assessment can facilitate identification of root cause analysis of the performance and further identification of improvements. In addition, a few researchers have started exploring the potential of employing systems thinking to sustainability assessment. Some of these works are presented in Section 4.2.
Moreover, the importance of existing sustainability assessment tools should not be overshadowed by the existing shortcomings. The systems thinking concept to be discussed below incorporates analysis, synthesis, and inquiry, which are all of the same importance. In the analysis, the system of interest is taken apart and is understood from the behavior of its components. The great number of existing sustainability assessment tools has contributed to the analysis of manufacturing organizations through the development of criteria, indicators, and metrics for different manufacturing components as well as for different sustainability aspects. An extensive amount of work has been done and what is needed now is to synthesize these achievements and develop a more unified solution through a systemic investigation.
4. Systems Thinking and Sustainability Assessment
In this section, the concept of systems thinking and its main ideas are presented (Section 4.1) as well as a review of research that focus on the application of systems thinking to sustainability assessment (Section 4.2). The appropriateness of the incorporation of systems thinking ideas into sustainability assessment is demonstrated.
4.1. Defining Thinking
Systems thinking is both a worldview and a process. It is beneficial both for understanding and developing a system, and for solving complex problems. Richmond [54] defined systems thinking as “the art and science to make reliable inferences about a behavior by developing an increasingly deep understanding of underlying structures”. A similar definition is given by Senge [55], “systems thinking is a discipline for seeing wholes... for seeing the ‘structures’ that underline complex situations”. Systems thinking can be presented through the three interrelated elements: synthesis, analysis, and inquiry (Figure 2). Important contributions to the development of systems thinking as a research field have been done by Ludwig von Bertalanffy, Ross Ashby, and Jay Forrester [56].
Figure 2.
The triple of systems thinking [57].
Figure 2.
The triple of systems thinking [57].
Sustainability assessment can benefit from the application of systems thinking ideas. In order to assess a manufacturing organization, sustainability assessment should capture the behavior of the organization. The manufacturing organization is a complex system that is more than the sum of the parts. Its emergent behavior is determined not by a behavior of each part but by the interactions between them. Systems thinking is an approach to understand an emergent behavior of a system by focusing on the whole, expressing it in terms of the interacting parts that contribute to the behavior. A system’s behavior is determined by the underlying structure of the system. However, in order to understand the behavior of manufacturing, one should look at organization as whole—consisting of employees, culture, processes, strategy, technologies, etc., with myriads of non-linear interconnections, delays, and feedbacks. Analysis of isolated parts and a structure only cannot provide understanding of the manufacturing behavior. Systems thinking acknowledges strong interactions between components of the system and unintended consequences that may result from these interactions. A behavior of a complex system is not a result of the linear cause and effect relations, but dynamic interactions between different systems’ components. Dynamic interactions are defined by (1) delays, i.e., differences in time between actions and its consequences; (2) non-linearity, i.e., one action can cause more than one consequence, and one consequence can be caused by more than one action; (3) feedback loops, both reinforcing and counteracting, i.e., the output from one node becomes its input [58,59]. If sustainability assessment is not able to depict such interactions, it can fail to highlight root causes of a current sustainability performance of manufacturing.
Corrective actions based on the snapshots of the system performance fail to address the dynamics of an organization. Senge [55] discussed real-life situations, which he calls “systems archetypes”, that can be treated with systems thinking approach. Such situations are (1) “fixes that fails”, i.e., a solution of the problem can cause other unintended consequences; (2) “limits to growth”, i.e., while actions that lead to success increase the demand for more actions, there are factors that can limit the growth; (3) “shifting the burden”, i.e., a solution applied in one part of organization may shift the problem to another part; (4) “tragedy of commons”, i.e., optimization of a sub-system can cause overload of the common resources; and (5) “accidental adversaries”, i.e., local optimization can cause a sub-optimization of a collective good. These situations cannot be identified with an assessment based on the snapshots, instead a study of manufacturing dynamics is required. Systems thinking ideas like feedbacks, delays, non-linear relationships may assist in this.
Multiple views is another idea of systems thinking that focuses on using multiple viewpoints instead of single viewpoint for studying a system’s behavior. Multiple viewpoints contribute to a greater understanding of the system’s behavior [60]. This idea of systems thinking can help to deal with the pluralism of sustainability assessment. According to Bond et al. [4], engagement of stakeholders and consideration of different framings of assessment problems and goals can enable integration of pluralism into sustainability assessment.
In addition, systems thinking can assist in identification of places where small changes can lead to big results. According to Senge, “systems thinking is a discipline … for discerning high from low leverage points” [55]. This aspect of systems thinking can enhance identification of actions toward sustainability.
4.2. Applying Systems Thinking to Sustainability Assessment
Researchers have been applying systems thinking, e.g., system dynamics, to sustainability-related issues for decades. However, an application of systems thinking to sustainability assessment has emerged slowly in recent years. According to Sterman [59], reductionism and analysis were tools of “machine age”, current “systems age” requires systems thinking and a holistic view on the world. A sustainability assessment is mostly built on the triple bottom line concept and Pressure-State-Response framework. However, these two approaches, which are based on the reductionist thinking, have shortcomings that have been already highlighted by different researchers. Pope et al. [6] point out that the separation of the sustainability concept into three pillars tends to emphasize potentially competing interests instead of focusing on the linkages and interdependencies between pillars. Moreover, Pope et al. suggest that a top-down approach to sustainability criteria generation should be adopted; sustainability, as a state to which society aims to, drives the definition of this state in terms of sustainability criteria. Kelly [61] criticized the Pressure-State-Response framework because information captured by it will tend to overlook important feedbacks and will lead to inefficient policies. Thus, Kelly emphasizes the importance of a systems thinking approach to decision making for sustainable development.
Due to the present shortcomings and challenges of sustainability assessment tools, a more holistic approach to the development of sustainability assessment should be applied. Skaržauskienė [62] points out that organization performance could be explained by competencies of systems thinking such as process orientation, interactivity, systems logic, and dynamic thinking. Several attempts have been made to incorporate systems thinking into sustainability assessment. Halog and Manik [63] developed an integrated methodological framework based on systems thinking by means of system dynamics, agent-based modeling, and network theory. The authors argue that the integrated system modeling method will become a basis for computational-based life cycle sustainability assessment of systems at various organizational levels. The proposed framework can aid in exploring various “what if” scenarios and advance the transition toward sustainability. Zhang et al. [64] adopted a systems thinking approach to examine the potential consequences of ad-hoc engineering decisions by modeling sustainable manufacturing problems. The authors developed causal loop diagrams for the enterprise, shop floor, and the operation levels, and concluded that a systems thinking approach can assist enterprises in making strategic, tactical, and operational decisions by providing a deep understanding of the manufacturing behavior. Uphoff [65] made an extensive analysis of systems thinking application to sustainability assessment of a food system. He stresses the importance of the system boundaries, identification of processes within the system of interest and relationships among the processes, and dimensions along and within the processes.
Apart from the conceptual research in applied systems thinking to sustainability assessment, a number of practical examples have been presented. One of them is a system-based approach to analyze the sustainability of biofuels supply chain that can enhance adaptive decision-making [66]. The authors argue that dynamic system modeling and simulation or agent-based modeling can address shortcomings of current sustainability assessments of biofuels supply chains. Another example is presented by Byomkesh and Blay-Palmer [67]. The researchers argue that many attempts that have been made to assess the sustainability of food and agriculture systems are questionable since they fail to address the interactions between sustainability dimensions and interrelationships between different life cycle phases. Thus, they proposed to incorporate systems thinking into the assessment of food and agriculture system sustainability. The introduced assessment framework based on systems thinking addresses a system complexity through modeling dynamic interactions among sustainability indicators. Systems thinking has also been applied to the sustainability assessment of wind energy [68]. System dynamics modeling was incorporated into analysis that is a first step to a simulator, which can help decision makers understand the dynamics associated with wind energy sustainability, while making informed decisions. Gibson [69] has recognized the need to consider socio-ecological system integrity. The author emphasize the importance of taking into account non-linearity of complex system behavior for sustainability assessment. Gibson et al. [70] applied integrated sustainability and resilience based assessments to a small-scale biodiesel project. A development of a systems description, depicting linkages within the operation and between the operation and the environment, was a part of a development of generic sustainability and resilience criteria.
Grace and Pope [23], who stressed the crucial role of systems thinking for sustainability assessments, presented a list of the differences that systems thinking applied to sustainability assessment can provide:
- Overly identify “enduring well-being” as the sustainability objective;
- Contextualize the role of the proposed actions in terms of the necessary transition to sustainability at the focal and global scale;
- Identify the causal relationships, feedback and delay that determine the dynamics of the focal socio-ecological system in the absence of the proposed action;
- Use this understanding to formulate alternative approaches to achieving the objectives of the proposed action;
- Identify the possible consequences of the alternatives;
- Formulate the preferred solution based on the enhanced understanding of system behavior;
- Guide the development of an adaptive management approach whereby system changes are monitored, the model is refined, and actions are revised in response.
The last but not least benefit provided by systems thinking is related to the identification of actions towards sustainability. Systems thinking can facilitate actions by means of leverage points. Leverage points are defined as places to intervene in a system such as numbers (taxes, subsidies), buffers, material stocks and flows, delays, positive and negative feedback loops, information flows, rules of the system, self-organization, goals of the system, the mindset out of which the goals, rules, structure and delays arise, and transcending paradigms [71]. Since sustainability assessment should direct decision makers in identification of actions towards sustainability [27], incorporation of systems thinking ideas into sustainability assessment can promote an identification of leverage points in manufacturing organization—i.e., places where a small shift in one thing can cause major changes in everything.
Overall, this review indicates that researchers have recognized the potential of systems thinking applied to the sustainability assessment domain. Practical examples demonstrate the advantages of sustainability assessments that incorporates systems thinking ideas. Although several advantages provided by systems thinking are essential, there is a need for more focused research in this area.
5. Development of Sustainability Assessment System Using the Systems Thinking
In this section, we propose a framework for development of sustainability assessment and highlight how each tool enables incorporation of systems thinking ideas into sustainability assessment. In addition, we present how the application of each tool can help decision makers overcome shortcomings and address challenges of sustainability assessment. The first inspiration for the development of a new sustainability assessment is the observed shortcomings. The second inspiration is the evidence of the potential of using systems thinking to address complex issues along with its ability to overcome limitations of existing sustainability assessment tools due to the application of the reductionist approach. The framework aims at the development of the sustainability assessment system, not a tool. While a sustainability assessment tool is any guideline, algorithm or software to be used, an assessment system also includes people who work with data collection and communicate with decision makers. The development of the sustainability assessment system should be systematic and address a holistic nature of the situation. Systems thinking provides different tools and methods from multiple areas such as operations research, systems engineering, system dynamics, decision analysis, management science, and business process reengineering. The development of a sustainability assessment system is proposed to employ the systems thinking perspective (Figure 3).
Since problem definition is an entry point to applied systems thinking [57], this is the first step of a development process. Analysis of multiple viewpoints is an essential part of systems thinking that helps to understand the problem and potential solutions. A conceptagon framework, proposed by Boardman et al. [72], is used for the ordered thinking about a solution to the problem—a new sustainability assessment system. These first three processes bring additional insights into the issue. After the issue of interest is studied in a systematic way, model-based systems engineering is applied to find a solution through a systematic investigation.
Figure 3.
The framework for development of the sustainability assessment system.
5.1. Problem Definition
Too often a problem statement actually addresses the symptom, rather than the root cause. Development of sustainability assessment tools often starts with the following problem: which criteria should be chosen for sustainability assessment? However, this is rather a symptom of the problem than the real problem. Thus, it is important to define the actual problem to be addressed by the development of a sustainability assessment system. In order to define the problem that the assessment system tends to solve, the following two questions should be answered: First, what is the sustainability assessment? Second, what is the purpose of sustainability assessment? Based on the reviewed definitions in Section 1, a sustainability assessment can be seen as a means that directs decision-making toward sustainability and helps to identify actions toward sustainability based on the specific information. Correspondingly, in the case of a manufacturing company, one of the purposes of an assessment is to support decision makers, facilitating the identification of actions that they should undertake in attempt to contribute to sustainable development. The main problem is to identify actions that will lead to sustainability—that is, what is wanted. Therefore, decisions are required, and the information is needed. In addition, the result of actions changes the information. Therefore, the problem that the sustainability assessment system tends to address is which information can truly lead to the identification of the right actions toward sustainability (Figure 4).
Figure 4.
The focus of the sustainability assessment system.
The challenge is to choose appropriate criteria for the assessment. Which criteria should be chosen in order to provide the information needed? A vast amount of criteria, indicators, and metrics has been proposed, yet there is presently no unified understanding or no unified criteria of sustainability. Many practitioners focus on sustainability performance of manufacturing and consider sustainability assessment tools as a means to find out. Then the question is how a sustainability assessment applied to an organization can contribute to global sustainability? Nooteboom [73] points out that each system to survive in a large environment must keep its wholeness as well as its “partness” of a larger system. This means that the sustainability assessment of the manufacturing organization should contribute to global sustainability as well as keeping itself sustainable. A manufacturing organization should then be considered as a sub-system that contributes to global sustainability rather than focusing only on its own performance. Grace and Pope [23] question whether a city, region, or nation can achieve sustainability in its own right, and whether the policy or project can achieve sustainability in its own right. The authors then argue that this is the central question to address in the development of sustainability assessments. Thus, the purposes of an assessment can be defined as to support decision makers, facilitating the identification of actions that they should undertake in attempt to contribute to global sustainable development. Then, the problem which sustainability assessment tends to address and can be defined in the following way:
“Which information can truly lead to the identification of actions that will contribute to global sustainable development?”
5.2. Analysis of Multiple Viewpoints
Analysis of multiple viewpoints can be approached through consideration of stakeholders perspectives and analysis from the perspective of different disciplines, e.g., scientist perspective, technologist, businessman, and sociologist [57].
Stakeholders and their characteristics have been discussed for decades [74]. According to Freeman [75], a stakeholder is defined as “any group or individual who can affect or is affected by the achievement of the organization’s objectives”. The identification of stakeholder viewpoints has been recognized as an essential activity for development of sustainability assessments, e.g., [76,77]. Each stakeholder is likely to have different viewpoints representing perception of the problem and the solution. In order to ensure that the system being developed—a sustainability assessment—meets the needs and expectations of all stakeholders, it is critical to understand different viewpoints and manage any conflicts and dissimilarities. Triste et al. [78], who analyzed the development process of a sustainability assessment tool, argue that “insufficient involvement of stakeholders and end users during the development process can lead to failure in the practical use of the tools”. The authors conclude with the list of actions which are success factors for tool adoption—“learning from stakeholders and end users, providing coaching for appropriate tool use, and structuring development of different tool types and exploring spin-offs from existing tools”. Similarly, Whitmarsh and Nykvist [79] state that a consideration of the diverse stakeholders’ perspectives is more likely to improve the validity and applicability of the assessment. Stakeholders’ engagement in the development of the sustainability assessment system is realized as an opportunity for adapting the system [80]. A unified list of the stakeholders for the sustainability assessment system does not exist. Only few researchers focus on the identification of the stakeholders for sustainability assessment, e.g., Lützkendorf et al. [81] analyzed the main stakeholders for sustainability assessment methods of buildings, and Mathur et al. [77] presented the generic stakeholder categories for the assessment of urban sustainability.
In case of development of the sustainability assessment system for manufacturing organization, stakeholders are those who can affect or be affected by the development of the assessment system as well as those who can affect or be affected by the use of the system. The primary stakeholders affecting the development process are the developers, research community, and manufacturing organizations. The stakeholder who can affect the use of the assessment tool is mainly the manufacturing organization, while those who can be affected by the use of the assessment system are manufacturing organizations, the world community, and the nature [82]. It can be concluded that involvement of manufacturing organizations in the development process may enhance the use of the sustainability assessment, while the involvement of research community may enhance the efficiency of the assessment system.
Moreover, analysis of multiple viewpoints can help to integrate pluralism into sustainability assessment. Bond et al. [4], who stated that “pluralism is the only way to deal with competing frames in sustainability assessment”, suggest that engagement of stakeholders and consideration of different framings of assessment problems and goals can enable integration of pluralism into sustainability assessment.
5.3. Conceptagon
Conceptagon is an analytical framework that can be used for (ordered) exploration of the system of interest or the problem situation (Figure 5). The framework was developed by Boardman et al. [72], and a few case studies have been conducted [83,84]. This framework facilitates the application of “systems thinking” to the development of the “sustainability assessment system”. The conceptagon is used to come to a coherent design of the assessment system which can efficiently support decision makers. According to the developers of the framework, there are no prescriptive methods to use the framework, and the user can choose any order. Thus, the order in which triples are presented herein is subjective. The conceptagon is applied to get a deeper understanding of the sustainability assessment system and its purpose. It also helps to apply ideas of systems thinking such as emergence, relationships/interconnectedness, feedbacks, nonlinearity, delays, and “system of systems”.
Figure 5.
The conceptagon (adapted from [72]).
Figure 5.
The conceptagon (adapted from [72]).
5.3.1. Boundary, Interior, Exterior
The definition of system boundaries and interfaces at the system boundaries are important at the early stage of the development process. A development project must narrow focus to a manageable number of problems that can be solved within the timeframe and resources available. On the other hand, sustainability is a holistic approach and exclusion of significant issues may affect the ability of an assessment system to address the entire area at hand. To deal with this situation, the boundaries of the sustainability assessment system should be set up according to time and resources available for a project without compromising the holistic nature of sustainability. The Seven Samurai framework, proposed by Martin [85], helps to deal with the complexity of 7 interrelated systems which constitute the design space (see Figure 6). The Seven Samurai framework is used to visualize interfaces between the sustainability assessment system being developed and exterior—other systems or elements of the environment.
Figure 6.
The Seven Samurai [85].
Figure 6.
The Seven Samurai [85].
The context system (S1) is a manufacturing company, and the problem (P1) is to provide information that can truly lead to identification of actions, which contributes to the global sustainable development. Sustainability assessment is a so-called intervention system (S2), which tends to address this problem. A realization system (S3) brings the intervention system into being. The realization system consists of all resources required to develop the sustainability assessment system. The main goal of the realization system is to get a deep understanding of the context system (S1)—manufacturing organization—where the problem occurs. Some questions that should be answered are: What is the root cause of the problem? Why is it a problem to identify actions toward sustainability? First, there is no agreed definition of sustainability. Thus, it is not easy to take actions toward something that is not clearly defined or has a different meaning for different actors. Second, to take actions toward sustainability, information about current performance and expected performance after actions is important. Obtaining this information is not trivial due to the complexity of manufacturing organizations.
The intervention system (S2) becomes a deployed system (S4) which may differ from the system that the realization system tends to develop because the intervention system will be transferred into a new complex environment. Thus, it is crucial for the realization system to understand a modified context system (S1’) that will contain a deployment system. Some important issues that require consideration are: Who will be the part of the sustainability assessment system? Who will interact with the sustainability assessment system? What kind of additional work does it create for the company? What kind of changes may occur in the company due to an application of the sustainability assessment? What does a company need to change in order to use the sustainability assessment system? When the sustainability assessment will be used by a company, it may also cause some new unknown problems for a company. These issues should be taken into consideration during the development process. When a newly deployed system—the sustainability assessment—is placed into the company, it will usually collaborate with some existing systems. These collaborating systems (S5) may consist of information systems (e.g., ERP, SCADA, MES) that contain data required for sustainability assessment. In addition, the sustainability assessment may provide information and interact with the decision-making system. Hence, developers should consider possible collaborating systems and the ways of collaboration: How will the sustainability assessment collaborate with information systems? In what way will the sustainability assessment system provide information for the decision-making system? Besides, there are competing systems (S7) which may in turn address the same problem as a deployed system. These are existing sustainability assessment tools that focus on the same problem. It is important to be familiar with such systems, to know their strong and weak sides. This ensures that the research is built upon the existing knowledge and lessons learned from previous research are taken into account.
A sustainment system (S6) aims at sustaining a deployed system—sustainability assessment, once it is adopted by manufacturing. For technical systems, sustainment means different enablers such as energy, training, maintenance, refurbishment, and waste removal. A sustainability assessment, however, is not a technical system, meaning that e.g., training procedures become increasingly more important than maintaining physical parts. Thus, the realization system may need to develop this training procedure and consider other sustainment options.
The Seven Samurai enhances the understanding of the exterior of the sustainability assessment system and enables the definition of boundaries. It is important to keep in mind that boundaries define the scope of interest and the area of responsibility. In the case of the sustainability assessment system, it is important to define parts of organization and sustainability issues that will be addressed. One of the significant decisions that should be made is whether the assessment system focuses on the organization only or on the relationships with its stakeholders (e.g., suppliers, customers) as well.
5.3.2. Inputs, Outputs, Transformation
After system boundaries are set up, the inputs and outputs, which are outside of the system boundaries, should be identified. The primary inputs to the sustainability assessment system are data from the company. However, the information about actions defined by decision makers should be included into the assessment system since this allows the follow-up activities. The identification of inputs that can provide the desired output—information that leads to actions toward global sustainability—is one of the challenges of the development process. The output of the assessment system is defined by the purpose of the system. Transformation is a second challenge of the development process—how to transform data into needed information avoiding both reductionism and subjectivity.
5.3.3. Wholes, Parts, Relationships
The reductionist approach to sustainability assessment, which is based on the selection and aggregation of indicators, has been criticized because of insufficient attention to linkages or relationships among the indicators [61]. According to Kelly [61], who studied the decisive information for sustainable development, most of the indicator frameworks can capture information about the current condition and the distance to the goal. However, indicators do not capture the information needed to make decisions about appropriate actions. Consideration of parts, wholes, and relationships is a strategy that can overcome this shortcoming. It is thus essential to have information about linkages or relationships. The assessment system should address elements of the manufacturing organization, nonlinear relationships between them, delays, and feedbacks; hence, the wholes can be addressed. In other words, the assessment system should have parts responsible for assessing the manufacturing elements and the relationships between them, as well as parts responsible for different sustainability issues and their relationships. In this case, the information about the structure and behavior of the system in which decision should be made, can be captured. Kelly [61] stressed that identification of the linkages, feedbacks, and system structure is critical for developing effective policies to support sustainable development.
5.3.4. Structure, Function, Process
The definition of the structure, function, and process enables understanding of the system. The function defines the system behavior; the process defines the sequence of activities needed to fulfill the function; and the structure defines system components and their relationships [57]. Based on the purpose of the sustainability assessment system defined in Section 5.1, the function of the assessment system is to provide information to decision makers, which can truly lead to the identification of actions that will contribute to global sustainable development. The process of the assessment describes the sequence of activities that leads to the generation of appropriate information for decision makers. The structure of the assessment system in turn defines parts of the system such as people, software and its components.
5.3.5. Command, Control, Communication
Figure 7 illustrates the application of command, control, and communication to the sustainability assessment system. First, the sustainability assessment system should communicate with decision makers to provide information that can lead to the identification of actions toward sustainability. Second, the decision makers command to the organization—give the order to take the action. Then, the assessment system should control the performance of manufacturing organization after the action is taken—whether the action led to sustainable development and whether it caused any changes.
Figure 7.
Command, control, communication and the sustainability assessment system.
5.3.6. Variety, Parsimony, Harmony
The development of the sustainability assessment system should seek for harmony reflected by two system’s concepts: parsimony and variety. Parsimony can be described as the search for the simplest explanation of the phenomena. In other words, it means that a developed system should be no more complex that it needs to be [86]. Variety, on the other hand, can be explained in the following way: “for a system to survive in its environment the variety of choices that the system is able to make must equal or exceed the variety of influences” [86]. Thus, to achieve harmony through the balance of parsimony and variety during the development, an assessment should have a holistic view on sustainability and manufacturing organization (variety). At the same time, sustainability assessment should not include too many sustainability criteria into assessment that requires an enormous amount of data (inputs) and complicated calculations (transformation)—parsimony. It is important to consider the fact that if the current performance or the result of actions is not accurately represented, it will likely lead to inappropriate decisions [61].
5.3.7. Emergence, Hierarchy, Openness
Openness is the measurement of the solidity of system boundaries and the level of interactions with the environment [57]. For the sustainability assessment system it can indicate the ability of a new component (e.g., a part of manufacturing, a sustainability criterion) to become included into the system. Manufacturing organization is a “system of systems” which consists of systems such product development, production, sales and marketing, and decision-making. It is important to address the hierarchical nature of the manufacturing organization as well as relationships between sub-systems that define the emergent behavior.
5.4. Model-Based Systems Engineering
Model-based systems engineering provides a framework for a system development lifecycle. It helps to manage a development lifecycle through the formalized application of modeling. According to theory, model-based systems engineering starts with the problem statement and system boundaries definition, and subsequently following the sequence of operations presented in Figure 8. The problem statement and system boundaries have been already discussed in previous sections (see Section 5.1 and Section 5.3.1).
Figure 8.
Model-based systems engineering process [87].
Figure 8.
Model-based systems engineering process [87].
5.4.1. Requirements Analysis
After analysis of multiple views, as presented in Section 5.2, stakeholders’ requirements are collected and analyzed. In the context of sustainability assessment, requirement analysis may help to balance needs of researchers and manufacturers. The requirements of researchers should be addressed, even though the assessment is being developed for a company, since researchers are those who focus on the sufficient methodology and those who are involved in the development process. This may help to remove the gap between the two communities, researchers and manufacturers. As discussed above, the use of sustainability assessment tools in manufacturing companies is limited because tools often fail to address effectively manufacturers’ requirements. For the sustainability assessment system, different types of requirements should be identified such as:
- Operational requirements—what is the purpose of the sustainability assessment system? (e.g., to provide information about issues which require improvements);
- Functional requirements—what should the system do? (e.g., address social aspect, address manufacturing processes, and address information flow);
- Interface requirements—how should the tool interact with external systems? (e.g., receive data from ERP automatically). This is addressed partly by Seven Samurai tool;
- Adaptability requirements—potential extensions and growth, e.g., ability to include additional sustainability criteria for assessment. This type of requirements refers to one of the conceptagon triple—openness, hierarchy, and emergence;
- Logistical requirements—sustainment, e.g., training of people.
Requirements of the manufacturing organization may be identified through structured interviews with case companies or a questionnaire, while requirements from researchers can be extracted from the literature. Different researchers’ requirements to the sustainability assessment system have been collected through the literature review and are summarized in Table 2. The requirements can be divided into two categories, those that deal with what should be addressed during the assessment and those that deal with how assessment should be done.
| What to consider during the assessment: |
| Address intergenerational equity |
| Address intragenerational equity |
| Address geographical equity |
| Address interspecies equity |
| Address procedural equity |
| Assess the system as a whole, including it parts and their interactions |
| Assess the system considering the different sustainability objectives together (integration) |
| Assess dynamics and interactions between trends and drivers of change |
| Adopt appropriate time horizon (short, medium, and long term) and (geographical) scope |
| Consider the normative nature of sustainability |
| Assessment of sustainability impacts and alternatives for decision-making, including synergies and trade-offs |
| Assessment is based on a conceptual sustainability framework and its indicators |
| Adapted to and integrated into the institutional context |
| Relations to global sustainability |
| Address socio-ecological system integrity |
| Address livelihood sufficiency and opportunity |
| Address resource maintenance and efficiency |
| Address socio-ecological civility and democratic governance |
| Address precaution and adaptation |
| Address immediate and long-term integration |
| Address resources consumption versus value creation |
| Triple bottom line consideration |
| Consider each product life cycle stage |
| How to do assessment: |
| Establish formal and transparent synergy/trade-off rules |
| Be responsive to change, including uncertainties and risks (dynamism) |
| Avoid irreversible risks and favors a precautionary approach |
| Mix of qualitative and quantitative metrics |
| Broad participation of stakeholders, including experts, while providing active leadership to the process |
| Develop and maintain adequate capacity |
| Continuous learning and improvement |
| Transparency regarding data (sources, methods), indicators, results, choices, assumptions, uncertainties, funding bodies and potential conflicts of interest |
| Ensure effective communications (clear language, fair and objective, visualization tools and graphics, make data appropriately available) |
| Iterative assessment process, starting at the onset of the decision-making process |
| Mix of leading and lagging indicators |
| Mix of absolute and relative metrics |
| Tradeoffs between external and internal performance measures |
| Integration of indicators across different policy arena |
| Reference values for indicators |
| Avoid “data drivenness” |
| Address interlinkages (socio-economic, socio-environmental, environmental-economic) |
5.4.2. Functional Behavior Analysis
The second step in the process focuses on what a new system must do and how well. The idea is to analyze a function of the main system that transforms inputs into outputs. Requirements defined in the previous stage determine the functional analysis. A focus on a functional behavior of the sustainability assessment system can enrich a development process. Advantages come from switching from the usual focus on physical implementation (which indicators to select, how many, how to aggregate them, etc.) to the focus on the logic of the assessment. Functional analysis should be based on the purpose of the assessment system, which is to support decision makers, facilitating the identification of actions that they should undertake in attempt to contribute to global sustainable development. In order to satisfy this purpose, the main function of the assessment system was defined in Section 5.3.4. This main function should then be broken down into sub-functions. This enables a focus on what kind of data should be collected and transformed into the information needed.
5.4.3. Architectural Synthesis
Only when a detailed analysis of functions and sub-functions of an assessment system is carried, a physical structure of the system should be developed. Since the objective of the assessment system is to assess a manufacturing company—a “system of systems”—the architecture of the assessment should address the elements of the “system of systems” as systems, subsystems, components, and interconnections. This brings two main challenges to the development process. The first challenge is to select sustainability criteria for manufacturing. Due to the high complexity of the sustainability concept, the application of complexity theory can be studied as a means to understand sustainability assessment. This may enable the identification of criteria. The second challenge is to choose the components of manufacturing for assessment due to the complexity and dynamism of a manufacturing organization. Thus, system dynamics, a tool for studying the dynamic behavior of complex systems, can be employed. A causal loop diagram is used for graphical representation of the dynamics of the system through feedbacks, stocks, flows, and time delays. Long and Scott [92], who studied the application of system dynamics to sustainable development and argued that sustainability is neither a state of the system nor is it a static goal or target to be achieved, concluded that system dynamics is the right tool to promote selection of moving targets for the system. Similarly, Kelly [61] stated that system dynamics can facilitate the identification of essential information about the structure and behavior of systems required for the development of effective actions towards sustainable development. Systems modeling tools are propounded as essential for effective sustainability assessment by Grace and Pope [23], arguing that sustainability assessment must consider global limits and that systems modeling can assist in doing so. System dynamics can be used to enhance the understanding of organization’s behavior and the development and examination of dynamic hypothesis. Thus, system dynamics models of a manufacturing organization can contribute to understanding of the organization’s behavior, and at the same time can be used for decision-making as a tool to test the consequences of the actions.
5.4.4. Validation and Verification
Validation and verification should be performed systematically during the development process. The verification process confirms that all elements of a new system perform according to the intended functions and meet the allocated requirements [87]. Verification ensures that all requirements are satisfied, i.e., behavior complies with requirements, and behavior is allocated to physical components and complies with requirements (Figure 9). In addition, the validation process tends to ensure that all requirements, logic, and architectural design are correct, complete, and consistent.
Figure 9.
Verification process (adopted from [93]).
Figure 9.
Verification process (adopted from [93]).
6. Conclusions
Many manufacturing organizations have problems to adapt sustainability assessment tools to specific business context and needs. One of the reasons is that developers of sustainability assessment tools focus on the tool by itself and not on its application capabilities. Thus, we emphasize the importance to consider the sustainability assessment system instead of assessment tool during the development process. Moreover, developers should realize that it is not enough to develop a good tool and simply place it in the company. It is important to realize that the applicability of an assessment tool can be increased by considering the environment in which the tool will be used.
One can argue that as long as a vast amount of sustainability assessment tools exists there is no need to develop a new one. However, the review of shortcomings presented herein illustrates the need and potential for improvements. Some researchers have also emphasized the need for a new approach to sustainability assessments. The limitations of existing sustainability assessments are associated with the underlying approach used for its development—reductionist. Thus, in this paper the need in utilizing systems thinking for the development of the sustainability assessment system is argued as a way to shift from analysis to the analysis-synthesis-inquiry triple. The application of systems thinking to sustainability assessment has already been propounded by a few researchers; hence, the objective of this paper was to show how developers of sustainability assessments may incorporate ideas of systems thinking into sustainability assessment of a manufacturing organization. After the actual need for a new sustainability assessment has been identified and studied, this paper discusses the potential for incorporating ideas of systems thinking into sustainability assessment. We proposed a framework for development of sustainability assessment and highlighted how the use of each tool i.e., analysis of multiple views, conceptagon, seven samurai, and model-based systems engineering, enables incorporation of systems thinking ideas into sustainability assessment, i.e.:
- Analysis of multiple viewpoints incorporates the idea of systems thinking of existence of multiple views on any system. In addition, analysis enables incorporation of pluralism into sustainability assessment.
- The conceptagon provides a structured and organized way to ensure that all significant aspects of the system are addressed e.g., emergence, relationships, boundaries, function, structure, purpose.
- Seven Samurai highlights interfaces between the sustainability assessment system and exterior, enriching a development process and may enhance the applicability of sustainability assessment.
- Model-based systems engineering assists the development process by helping to deal with complexity. The requirements analysis ensures that plurality of sustainability assessment is addressed. The functional analysis ensure that the focus is on the purpose of the assessment instead of on the means of it. The architectural analysis based on the system dynamics modeling can ensure that the ideas of systems thinking, which address the manufacturing dynamics, will be incorporated into sustainability assessment of a manufacturing organization.
The available evidence supports the proposition that systems thinking is a useful approach for development of the sustainability assessment system of manufacturing organizations. It helps to cope with, instead of omitting, the complexity of manufacturing and sustainability. Although Bond et al. [25] insist that no single, definitive and globally agreed sustainability assessment process can emerge, the proposed framework may assist the development of sustainability assessment by providing a systematic approach to address important aspects. The use of the framework brings to light many important aspects that often are neglected by developers:
- A clear definition of the problem that a particular sustainability assessment tends to address have to be given in the beginning of every development process.
- More attention should be paid to analysis of multiple viewpoints as a measure to overcome the gap between researchers and manufacturing and to address pluralism of sustainability assessment.
- A focus on the sustainability assessment system and its connections with exterior systems may increase applicability of the sustainability assessment in manufacturing.
- A clear understanding of the purpose and, thus, the output of sustainability assessment can enhance development of the architecture of the sustainability assessment and the choice of criteria for assessment.
- Sustainability assessment that captures dynamics of manufacturing can address situations that manufacturing may experience e.g., shift of a problem, sub-optimization, or solution that become a problem.
Acknowledgments
We would like to thank the reviewers for the valuable comments.
Author Contributions
All authors contributed significantly to the article, and have read and approved the final published manuscript.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Pope, J.; Bond, A.; Morrison-Saunders, A. A conceptual framework for sustainability assessment. In Handbook of Sustainability Assessment; Morrison-Saunders, A., Pope, J., Bond, A., Eds.; Edward Elgar: Northampton, UK, 2015; pp. 20–42. [Google Scholar]
- Bell, S.; Morse, S. Sustainability Indicators: Measuring the Immeasurable? Earthscan: Oxford, UK, 2008. [Google Scholar]
- Hopwood, B.; Mellor, M.; O’Brien, G. Sustainable development: Mapping different approaches. Sustain. Dev. 2005, 13, 38–52. [Google Scholar] [CrossRef]
- Bond, A.; Morrison-Saunders, A.; Howitt, R. Framework for comparing and evaluating sustainability assessment practice. In Sustainability Assessment: Pluralism, Practice and Progress; Routledge: London, UK, 2013; pp. 117–131. [Google Scholar]
- Ness, B.; Urbel-Piirsalu, E.; Anderberg, S.; Olsson, L. Categorising tools for sustainability assessment. Ecol. Econ. 2007, 60, 498–508. [Google Scholar] [CrossRef]
- Pope, J.; Annandale, D.; Morrison-Saunders, A. Conceptualising sustainability assessment. Environ. Impact Assess. Rev. 2004, 24, 595–616. [Google Scholar] [CrossRef]
- Videira, N.; Antunes, P.; Santos, R.; Lopes, R. A participatory modelling approach to support integrated sustainability assessment processes. Syst. Res. Behav. Sci. 2010, 27, 446–460. [Google Scholar] [CrossRef]
- Faulkner, W.; Badurdeen, F. Sustainable value stream mapping (sus-vsm): Methodology to visualize and assess manufacturing sustainability performance. J. Clean. Prod. 2014, 85, 8–18. [Google Scholar] [CrossRef]
- Singh, S.; Olugu, E.; Fallahpour, A. Fuzzy-based sustainable manufacturing assessment model for smes. Clean Technol. Environ. Policy 2014, 16, 847–860. [Google Scholar] [CrossRef]
- Zhang, H.; Haapala, K.R. Integrating sustainable manufacturing assessment into decision making for a production work cell. J. Clean. Prod. 2014, 105, 52–63. [Google Scholar] [CrossRef]
- Chen, D.; Thiede, S.; Schudeleit, T.; Herrmann, C. A holistic and rapid sustainability assessment tool for manufacturing smes. CIRP Ann.—Manuf. Technol. 2014, 63, 437–440. [Google Scholar] [CrossRef]
- Chen, D.; Schudeleit, T.; Posselt, G.; Thiede, S. A state-of-the-art review and evaluation of tools for factory sustainability assessment. Procedia CIRP 2013, 9, 85–90. [Google Scholar] [CrossRef]
- Rosen, M.A.; Kishawy, H.A. Sustainable manufacturing and design: Concepts, practices and needs. Sustainability 2012, 4, 154–174. [Google Scholar] [CrossRef]
- Paju, M.; Heilala, J.; Hentula, M.; Heikkil, A.; Johansson, B.; Leong, S.; Lyons, K. Framework and Indicators for a Sustainable Manufacturing Mapping Methodology. In Proceedings of the 2010 Winter Simulation Conference, Baltimore, Maryland, 5–8 December 2010; pp. 3411–3422.
- Hasna, A.M. A review of sustainability assessment methods in engineering. Int. J. Environ. Cult. Econ. Social Sustain. 2008, 5, 161–176. [Google Scholar]
- Moneim, A.F.A.; Galal, N.M.; Shakwy, M.E. Sustainable Manufacturing Indicators. In Proceedings of the Global Climate Change: Biodiversity and Sustainability, Alexandria, Egypt, 15–18 April 2013.
- Feng, S.C.; Joung, C.B. An Overview of a Proposed Measurement Infrastructure for Sustainable Manufacturing. In Proeedings of the 7th Global Conference on Sustainable Manufacturing, Chennai, India, 2–4 December 2009; p. 12.
- Poveda, C.A.; Lipsett, M.G. A review of sustainability assessment and sustainability/environmental rating systems and credit weighting tools. J. Sustain. Dev. 2011, 4, 36–55. [Google Scholar] [CrossRef]
- Singh, R.K.; Murty, H.R.; Gupta, S.K.; Dikshit, A.K. An overview of sustainability assessment methodologies. Ecol. Indic. 2012, 15, 281–299. [Google Scholar] [CrossRef]
- Ioppolo, G.; Cucurachi, S.; Salomone, R.; Saija, G.; Ciraolo, L. Industrial ecology and environmental lean management: Lights and shadows. Sustainability 2014, 6, 6362–6376. [Google Scholar] [CrossRef]
- Deutz, P.; Ioppolo, G. From theory to practice: Enhancing the potential policy impact of industrial ecology. Sustainability 2015, 7, 2259–2273. [Google Scholar] [CrossRef]
- Moldavska, A.; Welo, T. On the applicability of sustainability assessment tools in manufacturing. Procedia CIRP 2015, 29, 621–626. [Google Scholar] [CrossRef]
- Grace, W.; Pope, J. A systems approach to sustainability assessment. In Handbook of Sustainability Assessment; Morrison-Saunders, A., Pope, J., Bond, A., Eds.; Edward Elgar: Northampton, UK, 2015; pp. 285–320. [Google Scholar]
- Cashmore, M.A.; Kørnøv, L. The changing theory of impact assessment. In Sustainability Assessment: Pluralism, Practice and Progress; Bond, A., Morrison-Saunders, A., Howitt, R., Eds.; Routledge: London, UK, 2013; pp. 18–33. [Google Scholar]
- Bond, A.; Pope, J.; Morrison-Saunders, A. Introducing the roots, evolution and effectiveness of sustainability assessment. In Handbook of Sustainability Assessment; Morrison-Saunders, A., Pope, J., Bond, A., Eds.; Edward Elgar: Northampton, UK, 2015; pp. 3–19. [Google Scholar]
- Marsden, G.; Kimble, M.; Nellthorp, J.; Kelly, C. Sustainability assessment: The definition deficit. Int. J. Sustain. Transp. 2010, 4, 189–211. [Google Scholar] [CrossRef]
- Bond, A.J.; Morrison-Saunders, A. Re-evaluating sustainability assessment: Aligning the vision and the practice. Environ. Impact Assess. Rev. 2011, 31, 1–7. [Google Scholar] [CrossRef]
- Hugé, J.; Waas, T.; Eggermont, G.; Verbruggen, A. Impact assessment for a sustainable energy future—Reflections and practical experiences. Energy Policy 2011, 39, 6243–6253. [Google Scholar] [CrossRef]
- Devuyst, D.; Hens, L.; de Lannoy, W. How Green is the City?: Sustainability Assessment and the Management of Urban Environments; Columbia University Press: New York, NY, USA, 2001; p. 457. [Google Scholar]
- Langeveld, H.; Sanders, J.; Meeusen, M. The Biobased Economy: Biofuels, Materials and Chemicals in the Post-Oil Era; Routledge: London, UK, 2012. [Google Scholar]
- Wiek, A.; Binder, C. Solution spaces for decision-making—A sustainability assessment tool for city-regions. Environ. Impact Assess. Rev. 2005, 25, 589–608. [Google Scholar] [CrossRef]
- Hugé, J.; Waas, T.; Dahdouh-Guebas, F.; Koedam, N.; Block, T. A discourse-analytical perspective on sustainability assessment: Interpreting sustainable development in practice. Sustain. Sci. 2013, 8, 187–198. [Google Scholar] [CrossRef]
- Hacking, T.; Guthrie, P. A framework for clarifying the meaning of triple bottom-line, integrated, and sustainability assessment. Environ. Impact Assess. Rev. 2008, 28, 73–89. [Google Scholar] [CrossRef]
- Waas, T.; Hugé, J.; Block, T.; Wright, T.; Benitez-Capistros, F.; Verbruggen, A. Sustainability assessment and indicators: Tools in a decision-making strategy for sustainable development. Sustainability 2014, 6, 5512–5534. [Google Scholar] [CrossRef] [Green Version]
- Joung, C.B.; Carrell, J.; Sarkar, P.; Feng, S.C. Categorization of indicators for sustainable manufacturing. Ecol. Indic. 2013, 24, 148–157. [Google Scholar] [CrossRef]
- Fan, C.; Carrell, J.D.; Zhang, H.-C. An Investigation of Indicators for Measuring Sustainable Manufacturing. In Proceedings of the 2010 IEEE International Symposium on Sustainable Systems and Technology (ISSST), Arlington, VA, USA, 17–19 May 2010; pp. 1–5.
- Veleva, V.; Ellenbecker, M. Indicators of sustainable production: Framework and methodology. J. Clean. Prod. 2001, 9, 519–549. [Google Scholar] [CrossRef]
- NIST. Sustainable manufacturing indicators repository. Available online: http://www.mel.nist.gov/msid/SMIR/ (accessed on 26 December 2015).
- Taisch, M.; Sadr, V.; May, G.; Stahl, B. Sustainability assessment tools—State of research and gap analysis. In Advances in Production Management Systems. Sustainable Production and Service supply Chains; Prabhu, V., Taisch, M., Kiritsis, D., Eds.; Springer: Berlin, Germany, 2013; Volume 415, pp. 426–434. [Google Scholar]
- Sikdar, S.K. Sustainable development and sustainability metrics. AIChE J. 2003, 49, 1928–1932. [Google Scholar] [CrossRef]
- Clancy, G.; Fröling, M.; Svanström, M. Insights from guiding material development towards more sustainable products. Int. J. Sustain. Des. 2013, 2, 149–166. [Google Scholar] [CrossRef]
- Ameta, G.; Rachuri, S.; Fiorentini, X.; Mani, M.; Fenves, S.; Lyons, K.; Sriram, R. Extending the notion of quality from physical metrology to information and sustainability. J. Intell. Manuf. 2011, 22, 737–750. [Google Scholar] [CrossRef]
- Brown, A.; Amundson, J.; Badurdeen, F. Sustainable value stream mapping (sus-vsm) in different manufacturing system configurations: Application case studies. J. Clean. Prod. 2014, 85, 164–179. [Google Scholar] [CrossRef]
- Lanz, M.; Järvenpää, E.; Nylund, H.; Tuokko, R.; Torvinen, S.; Georgoulias, K. Sustainability and performance indicators landscape. In Proceedings of the 24th International Conference on Flexible Automation and Intelligent Manufacturing, San Antonio, TX, USA, 20–23 May 2014; p. 8.
- Spitzer, D.R. Transforming Performance Measurement: Rethinking the Way We Measure and Drive Organizational Success; American Management Association: New York, NY, USA, 2007. [Google Scholar]
- Hák, T.; Moldan, B.; Dahl, A.L. Sustainability Indicators: A scientific Assessment; Island Press: Washington, DC, USA, 2012. [Google Scholar]
- Fantini, P.; Palasciano, C.; Taisch, M. Back to intuition: Proposal for a performance indicators framework to facilitate eco-factories management and benchmarking. Procedia CIRP 2015, 26, 1–6. [Google Scholar] [CrossRef]
- Snodgrass, D. A Note on Indicators of Sustainability for Value Chain Projects; United States Agency for International Development: Washing, DC, USA, 2012. [Google Scholar]
- Andrews, E.S.; Barthel, L.-P.; Beck, T.; Benoît, C.; Ciroth, A.; Cucuzzella, C.; Gensch, C.-O.; Hébert, J.; Lesage, P.; Manhart, A.; et al. Guidelines for Social Life Cycle Assessment of Products; UNEP: Nairobi, Kenya, 2009; p. 104. [Google Scholar]
- Gibson, R.B. Beyond the pillars: Sustainability assessment as a framework for effective integration of social, economic and ecological considerations in significant decision-making. J. Environ. Assess. Policy Manag. 2006, 8, 259–280. [Google Scholar] [CrossRef]
- Fischer, J.; Weinert, N.; Herrmann, C. Method for selecting improvement measures for discrete production environments using an extended energy value stream model. Procedia CIRP 2015, 26, 133–138. [Google Scholar] [CrossRef]
- Despeisse, M.; Oates, M.R.; Ball, P.D. Sustainable manufacturing tactics and cross-functional factory modelling. J. Clean. Prod. 2013, 42, 31–41. [Google Scholar] [CrossRef]
- Despeisse, M.; Ball, P.D.; Evans, S.; Levers, A. Industrial ecology at factory level—A conceptual model. J. Clean. Prod. 2012, 31, 30–39. [Google Scholar] [CrossRef] [Green Version]
- Richmond, B. Systems thinking/system dynamics: Let’s just get on with it. Syst. Dyn. Rev. 1994, 10, 135–157. [Google Scholar] [CrossRef]
- Senge, P.M. The Fifth Discipline: The Art and Practice of the Learning Organization; Doubleday/Currency: New York, NY, USA, 1990. [Google Scholar]
- Klir, G. Facets of Systems Science; Springer US: New York, NY, USA, 2013. [Google Scholar]
- Edson, R. Systems Thinking. Applied. A Primer; Applied Systems Thinking (ASYST) Institute, Analytic Services Inc.: Arlington, VA, USA, 2008. [Google Scholar]
- Gharajedaghi, J. Systems Thinking: Managing Chaos and Complexity: A platform for Designing Business Architecture; Elsevier Science: Philadelphia, PA, USA, 2011. [Google Scholar]
- Sterman, J. Business Dynamics: Systems Thinking and Modeling for a Complex World; Irwin/McGraw-Hill: New York, NY, USA, 2000. [Google Scholar]
- Saha, P. A Systemic Perspective to Managing Complexity with Enterprise Architecture; IGI Global: Hershey, PA, USA, 2013. [Google Scholar]
- Kelly, K.L. A systems approach to identifying decisive information for sustainable development. Eur. J. Oper. Res. 1998, 109, 452–464. [Google Scholar] [CrossRef]
- Skaržauskienė, A. Managing complexity: Systems thinking as a catalyst of the organization performance. Meas. Bus. Excell. 2010, 14, 49–64. [Google Scholar]
- Halog, A.; Manik, Y. Advancing integrated systems modelling framework for life cycle sustainability assessment. Sustainability 2011, 3, 469. [Google Scholar] [CrossRef]
- Zhang, H.; Calvo-Amodio, J.; Haapala, K.R. A systems thinking approach for modeling sustainable manufacturing problems in enterprises. In Proceedings of the International Annual Conference (IAC) of the American Society for Engineering Management (ASEM), Minneapolis, MI, USA, 3–5 October 2013.
- Uphoff, N. Systems thinking on intensification and sustainability: Systems boundaries, processes and dimensions. Curr. Opin. Environ. Sustain. 2014, 8, 89–100. [Google Scholar] [CrossRef]
- Halog, A.; Awuah, N. The Need for Integrated Life Cycle Sustainability Analysis of Biofuel Supply Chains; INTECH Open Access Publisher: Rijeka, Croatia, 2013. [Google Scholar]
- Byomkesh, T.; Blay-Palmer, A. Incorporating system thinking in assessments of food and agriculture system sustainability. In Graduate Student Workshop of Waterloo Food Issues Group; WatFIG: Waterloo, ON, Canada, 2013. [Google Scholar]
- Tejeda, J.; Ferreira, S. Applying systems thinking to analyze wind energy sustainability. Procedia Comput. Sci. 2014, 28, 213–220. [Google Scholar] [CrossRef]
- Gibson, R.B.; Hassan, S. Sustainability Assessment: Criteria and Processes; Earthscan: Oxford, UK, 2005; p. 254. [Google Scholar]
- Gaudreau, K.; Gibson, R.B. Illustrating integrated sustainability and resilience based assessments: A small-scale biodiesel project in barbados. Impact Assess. Proj. Apprais. 2010, 28, 233–243. [Google Scholar] [CrossRef]
- Meadows, D.; Wright, D. Thinking in Systems: A Primer; Chelsea Green Publishing: Hartford, VT, USA, 2008. [Google Scholar]
- Boardman, J.; Sauser, B.; John, L.; Edson, R. The Conceptagon: A Framework for Systems Thinking and Systems Practice. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Antonio, TX, USA, 11–14 October 2009; pp. 3299–3304.
- Nooteboom, S. Impact assessment procedures for sustainable development: A complexity theory perspective. Environ. Impact Assess. Rev. 2007, 27, 645–665. [Google Scholar] [CrossRef]
- Mitchell, R.K.; Agle, B.R.; Wood, D.J. Toward a theory of stakeholder identification and salience: Defining the principle of who and what really counts. Acad. Manag. Rev. 1997, 22, 853–886. [Google Scholar]
- Freeman, R.E. Strategic Management: A Stakeholder Approach; Cambridge University Press: Cambridge, UK, 2010. [Google Scholar]
- Lützkendorf, T.; Hájek, P.; Lupíšek, A.; Immendörfer, A.; Nibel, S.; Häkkinen, T. New trends in sustainability assessment systems—Based on top-down approach and stakeholders needs. Int. J. Sustain. Build. Technol. Urban Dev. 2012, 3, 256–269. [Google Scholar] [CrossRef]
- Mathur, V.N.; Price, A.D.F.; Austin, S.A.; Moobela, C. Defining, Identifying and Mapping Stakeholders in the Assessment of Urban Sustainability. In Proceedings of the SUE-MOT International Conference on Whole Life Urban Sustainability and Its Assessment, Glasgow, Scotland, 27–29 June 2007; Horner, M., Hardcastle, C., Price, A., Bebbington, J., Eds.; p. 18.
- Triste, L.; Marchand, F.; Debruyne, L.; Meul, M.; Lauwers, L. Reflection on the development process of a sustainability assessment tool: Learning from a flemish case. Ecol. Soc. 2014, 19. Article 47. [Google Scholar] [CrossRef]
- Whitmarsh, L.; Nykvist, B. Integrated sustainability assessment of mobility transitions: Simulating stakeholders’ visions of and pathways to sustainable land-based mobility. Int. J. Innov. Sustain. Dev. 2008, 3, 115–127. [Google Scholar] [CrossRef]
- Berardi, U. Chapter 15—Sustainability assessments of buildings, communities, and cities. In Assessing and Measuring Environmental Impact and Sustainability; Klemeš, J.J., Ed.; Butterworth-Heinemann: Oxford, UK, 2015; pp. 497–545. [Google Scholar]
- Lützkendorf, T.; Balouktsi, M.; Immendörfer, A.; Nibel, S.; Bosdevigie, B.; Lebert, A.; Fies, B.; Iñarra, P.H.; Lupíšek, A.; Hajek, P.; et al. Sustainability and Performance Assessment and Benchmarking of Buildings; VTT: Espoo, Finland, 2012. [Google Scholar]
- Driscoll, C.; Starik, M. The primordial stakeholder: Advancing the conceptual consideration of stakeholder status for the natural environment. J. Bus. Ethics 2004, 49, 55–73. [Google Scholar] [CrossRef]
- McGee, S.; Edson, R. Extending the conceptagon as an analytic framework: A case study of public preparedness in israel. In Proceedings of the 8th Conference on Systems Engineering Research, Hoboken, NJ, USA, 17–19 March 2010.
- McGee, S.; Joel, M.; Edson, R. Mexico’s Cartel Problem: A Systems Thinking Perspective; Applied Systems Thinking Institute, Analytic Services, Inc.: Arlington, VA, USA, 2011. [Google Scholar]
- Martin, J.N. The seven samurai of systems engineering: Dealing with the complexity of 7 interrelated systems. In Proceedings of the INCOSE International Symposium, Toulouse, France, 20–24 June 2004; Volume 14, pp. 459–470.
- Boardman, J.; Sauser, B. Systems Thinking: Coping with 21st Century Problems; CRC Press: Boca Raton, FL, USA, 2008. [Google Scholar]
- Long, D.; Scott, Z. A Primer for Model-Based Systems Engineering; Vitech Company: Blacksburg, VA, USA, 2011. [Google Scholar]
- Fiksel, J.; McDaniel, J.; Mendenhall, C. Measuring Progress towards Sustainability Principles, Process, and Best Practices. In Proceedings of the Greening of Industry Network Conference Best Practice Proceedings, Chapell Hill, NC, USA, 14–17 November 1999.
- Ahi, P.; Searcy, C. An analysis of metrics used to measure performance in green and sustainable supply chains. J. Clean. Prod. 2015, 86, 360–377. [Google Scholar] [CrossRef]
- Meyer, M.W. Rethinking Performance Measurement: Beyond the Balanced Scorecard; Cambridge University Press: Cambridge, UK, 2003. [Google Scholar]
- Pintér, L.; Hardi, P.; Martinuzzi, A.; Hall, J. Bellagio stamp: Principles for sustainability assessment and measurement. Ecol. Indic. 2012, 17, 20–28. [Google Scholar] [CrossRef]
- Hjorth, P.; Bagheri, A. Navigating towards sustainable development: A system dynamics approach. Futures 2006, 38, 74–92. [Google Scholar] [CrossRef]
- Estefan, J.A. Survey of Model-Based Systems Engineering (MBSE) Methodologies; Incose MBSE Focus Group: Pasadena, CA, USA, 2007; p. 47. [Google Scholar]
© 2015 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons by Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Moldavska, A.; Welo, T. Development of Manufacturing Sustainability Assessment Using Systems Thinking. Sustainability 2016, 8, 5. https://doi.org/10.3390/su8010005
AMA Style
Moldavska A, Welo T. Development of Manufacturing Sustainability Assessment Using Systems Thinking. Sustainability. 2016; 8(1):5. https://doi.org/10.3390/su8010005
Chicago/Turabian StyleMoldavska, Anastasiia, and Torgeir Welo. 2016. "Development of Manufacturing Sustainability Assessment Using Systems Thinking" Sustainability 8, no. 1: 5. https://doi.org/10.3390/su8010005
Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.
