- freely available
Sustainability 2011, 3(2), 469-499; doi:10.3390/su3020469
Published: 23 February 2011
Abstract: The need for integrated methodological framework for sustainability assessment has been widely discussed and is urgent due to increasingly complex environmental system problems. These problems have impacts on ecosystems and human well-being which represent a threat to economic performance of countries and corporations. Integrated assessment crosses issues; spans spatial and temporal scales; looks forward and backward; and incorporates multi-stakeholder inputs. This study aims to develop an integrated methodology by capitalizing the complementary strengths of different methods used by industrial ecologists and biophysical economists. The computational methodology proposed here is systems perspective, integrative, and holistic approach for sustainability assessment which attempts to link basic science and technology to policy formulation. The framework adopts life cycle thinking methods—LCA, LCC, and SLCA; stakeholders analysis supported by multi-criteria decision analysis (MCDA); and dynamic system modelling. Following Pareto principle, the critical sustainability criteria, indicators and metrics (i.e., hotspots) can be identified and further modelled using system dynamics or agent based modelling and improved by data envelopment analysis (DEA) and sustainability network theory (SNT). The framework is being applied to development of biofuel supply chain networks. The framework can provide new ways of integrating knowledge across the divides between social and natural sciences as well as between critical and problem-solving research.
In order to advance the inter-/cross-/trans-disciplinary notions of industrial ecology, ecological/evolutionary economics, biophysical/thermo-economics, sustainability science and engineering in confronting development-related issues (e.g., global warming, energy security, rural and developing countries' development, poverty, etc.), we need to develop transformative, life cycle, systems-oriented thinking frameworks, tools and methods to support policy and regulation formulation as well as for corporate decision making. The urgency in developing these integrative tools and methods to support policies, regulations and practices for sustainable development both at national and international levels have been stressed in various sustainability-related studies [1-6]. Harmonization of existing assessment approaches is needed to consolidate methodologies, and enhance national capacity to implement integrated sustainability assessment. Though there are various terminologies used for assessment approaches such as strategic environmental assessment, sustainability impact assessment, sustainability assessment, and integrated assessment, all of these aim to achieve the overall objective of sustainable development [7,8]. In support of developing a sustainable industrial ecosystem, this article proposes an integrative framework that builds on the complementary strengths of various approaches and methodologies used by industrial ecologists, ecological and biophysical economists and engineers. Developing and using such a framework would help reduce the confusion from the multiplicity of terminologies and encourage collaborative efforts.
However, there are significant computational challenges in integrated sustainability assessment (ISA) for design of policies, regulations and practices for sustainable development [1,9]. These fall into the realm of computing and information science (e.g., optimization, data mining & analysis, artificial intelligence, dynamical models, etc.). An integrated systems modelling belongs to the emerging field of computational and quantitative sustainability. Computational sustainability (CS) aims to apply techniques from computer and information science (e.g., cloud computing) and related disciplines (e.g., operations research, management science) to the balancing of environmental, economic, and societal needs, in pursuit of sustainable development [10-12]. CS has similar agenda with industrial ecology which brings together computational fields and other fields (e.g., evolutionary, ecological, biophysical, thermo economics) which have emphasis on the study of sustainability-related problems. Through computational modelling and analysis of interacting sub-systems in developing a sustainable system, transformative concepts, system models and algorithms for science and engineering will emerge.
Sustainable development is a complex, multi-dimensional phenomenon, with a breadth and depth that cannot be fully covered by the current portfolio of reductionist-oriented tools. We therefore need a new generation of modelling tools that can (semi-) quantitatively assess at least the triple dimensions of sustainable development, in terms of multiple scales, multiple domains and multiple generations [2,13]. Besides the published works of Halog [1,14,15], the European MATISSE (Methods and Tools for ISA) proposes an assessment that involves a step-wise procedure that can advance the science and application of ISA to policies, regulations and business practices . Another related project is the SUE-MoT which is focused on developing holistic metrics, methods and tools in a way that reflects stakeholder values . SUE-MoT has been working to develop a toolkit that brings together many approaches allowing key decision-makers to identify the most appropriate for their own projects and to combine the results based on their values. Heijungs et al.  develop a general conceptual framework for sustainability analysis and insinuate the consideration of interrelationships and stakeholders involvement. However, they have failed to discuss the specific well-established methods for multi-criteria analysis and dynamic system modelling that can be used for technical, environmental, physical, and micro-economic modelling. In alignment with MATISSE, SUE-MOT and Heijungs et al. efforts, our study aims to address the interlinkages of system methods used by analyzing the complementary strengths and thus developing an integrated decision-support framework for assessing systems sustainability. This advanced framework can be aptly described as multi-criteria, systems perspective and engaging the relevant stakeholders. This methodological development is different from existing methods which are piecemeal, steady-state, reductionist approach to solving complex issues. A reductionist approach has been employed because most of us have been trained to handle a complicated issue by disaggregating it to smaller manageable parts, but oftentimes they become grossly oversimplified [16,17]. We thought that by simply adding the parts (e.g., current thinking for multi-disciplinary approach) together, we would be able to solve the problem. The weakness of this multi-approach is that we have forgotten to include the trans/cross-boundary impacts or interrelationships between and among the parts. We forgot to realize that the whole is not really equal to the sum of the parts [1,18] when considering the interconnections between sub-systems of our global society. Thus, besides finding out the local optima in optimization problems, we need to optimize the global optimal solutions (with consideration of interlinkages) for the benefit of the whole. Similar to the case of managing companies successfully, we need to realize that a company is connected or co-dependent with other players in the whole global supply/value chain network. Besides optimizing the individual players or agents' stakes, we need to engineer the whole system to make it sustainable and resilient over time. Unless we think beyond our respective silos or understand the issue beyond the confines of our disciplines, our global problems will persist. Accounting holistic interconnections are lacking in contemporary methods and therefore should be included in enhancing our existing approaches, methods, tools and framework. The interactions of different sub-systems at micro, meso and macro levels have not been modelled in many contemporary life cycle methods though Heijungs et al.  have insinuated this in their research paper. This paper would like to advance the development of an integrated methodological framework for modelling and eventually designing sustainable, resilient systems. Instead of pointing out the obvious individual weaknesses of current methodologies being used, we rather focus on their respective individual strengths to provide specific results and then capitalize their complementary strengths to overcome specific weaknesses of individual methods to produce a robust, comprehensive sustainability assessment results. By following the proposed integrative framework, we can assess or evaluate any systems both existing and emerging at any level over time in view of attaining our sustainable development or sustainability-relevant goals.
In succeeding pages, we discuss first the proposed integrated methodological framework for understanding the challenges of sustainable development, extracting stakeholders' voices, modelling the interconnections, and assessing sustainable systems using well-established and emerging methods. We will explain the usefulness of the individual methods with regard to developing an integrative, life cycle, systemic dynamic methodology. Then we discuss our on-going applications to bioenergy systems. Finally, we conclude the implications of using this integrated framework in assessing both existing and emerging systems with one goal—Global Sustainability.
2. Integrated Assessment for Systems Sustainability
In the recently concluded 2011 International Congress on Sustainability Science and Engineering in Arizona, USA, it has been argued that to make the notion of sustainable development useful and operational to most of stakeholders, 4 aspects have to converge. These include (1) science and technology must support it; (2) right policies and regulatory frameworks should be well-formulated; (3) businesses should be actively involved; and (4) public stakeholders must understand and support it either by incorporating their voices in the process and showing the results in understandable interactive manner (e.g., the use of dashboards). Sustainable development thus requires “sustainable human communities that act like natural ones, living within a natural ebb and flow of energy from the sun and plants. This requires redesigning all industrial, residential, and transportation systems so that everything we use springs easily from the earth and returns back to it” [1,19,20]. This challenging pursuit needs a shift or transition from domination to partnership which identifies interlinkages among sub-systems. Understanding these interconnections facilitate a transition from our existing financially-based economic system to one that strives to become fully compatible with ecosystem integrity. This transformative paradigm emphasizes the need for highest achievable levels of ecological efficiency in industrial activity while at the same time promoting quality, cooperation, and conservation [4,21]. Ecological integrity of human-engineered systems is central to interdisciplinary fields of ecological economics, industrial ecology, evolutionary and biophysical economics.
In view of addressing the triple pillars of sustainable development, below are brief discussions and description of existing methods with regard to developing an integrated sustainability assessment decision support system.
2.1. Environmental Dimension of Sustainability
The proliferation of carbon/greenhouse gas accounting and water footprinting methods  nowadays for assessing environmental performance of companies' products, which grossly oversimplify the environmental life cycle assessment (LCA), are considered insufficient [2,13,23] to understand the dynamic interrelationships between the environmental and ecological impacts as well as their implications for resource consumptions. The development of these simplified business-tailored approaches is based on traditional linear, piecemeal and reductionist mentality, which cannot eventually contribute to solving our existing urgent problems. These methods have been developed for business-as-usual convenience and perspectives, rather than to face the difficult truth in addressing the critical root causes (not symptoms) of environmental system problems. LCA is considered the most comprehensive approach for environmental impact, though there are challenges in acquiring quality data [2,24,25]. This issue in data acquisition can be addressed through virtual collaborations by adopting cyberinfrastructure technologies to connect and improve existing databases to become web-based [10,12].
There are 3 or more variations of LCA. The ISO-14000 or process-based LCA [26,27] is the most widely applied. Hendrickson et al  have developed EIO (Economic Input-Output)-LCA which is based on the economic input-output data (recently called environmentally extended economic input-output analysis). There are also other developed models which use EIO as a basis such as the mixed-unit IO model for environmental life-cycle assessment and material flow analysis for US  and Physical Input Monetary Output (PIMO) model for understanding material flows within ecological-economic systems . Heijungs and Suh  have suggested combining the process-based LCA and EIO-LCA to form a hybrid to address missing data and produce better LCA results. Another LCA methodological development is the ecologically based LCA (Eco-LCA) which is a framework to account for the role of ecosystem goods and services (e.g., water, mineral, carbon sequestration) in the life cycle of economic activities [31-33]. Classification schemes in Eco-LCA include, renewable versus nonrenewable, biotic versus abiotic, materials versus energy, or in terms of their originating ecosphere (lithosphere, biosphere, hydrosphere, atmosphere, and other services). Using any of these variations of LCA provides the environmental emissions, impacts and resource consumptions (hotspots or critical indicators or metrics) for a particular system considered. Materials flow analysis (MFA) [34-36], substance flow analysis (SFA) [37-40] and energy flow analysis (EFA)  can further provide additional material/substance/energy relevant hotspots.
2.2. Economic Dimension of Sustainability
The 2nd pillar of sustainability is the prosperity/profitability aspect. There has been a wide range of existing and established tools for estimating costs and revenues . Finding and estimating cost and economic related data is less than a challenge compared to environmental life cycle data.
Complementary to environmental LCA, “Environmental” Life Cycle Costing (LCC) is an assessment of all costs associated with the life cycle of a product system that are directly covered by any one or more of the actors in the product life cycle with inclusion of externalities that are anticipated to be internalized in the decision-relevant future [43,44]. Bubeck  has used this approach for early product development. This requires a structuring of the costs according to lifecycle stages where the focus is on money flows (similar to material and energy flows) and taking stakeholders' perspective into account. LCA and LCC represent two different ways of extracting indicators from exactly the same system . LCC can provide the critical economic indicators or metrics or hotspots that are relevant to the system considered.
2.3. Social Dimension of Sustainability
Though there are multiple dimensions in sustainability definition [14,46,47], the most popular one is the triple dimensions of sustainability/triple bottom line. The social life cycle assessment (SLCA) is the third dimension which accounts the impact of an organization, product or system on society [48,49]. The social benefits can be estimated by analyzing the effects of the stakeholders at local, national and global levels . The majority of social indicators measure the degree to which societal values and goals in the particular areas of life can be achieved. However, many social issues on which a performance measurement takes place are not easily quantifiable. A review and current challenges of SLCA have been published by Jørgensen et al. [51-53]. Recently, UNEP has also published Guidelines for SLCA of Products .
In contrast to environmental and economic aspects of sustainability assessment, social assessment still lacks a broad consensus on adequate indicators or a standardized method . Social impacts and benefits are weighted differently by different interest groups and in different countries and regions, and their evaluation is subject to swifter changes over time (e.g., cultural or economic change). Eight indicators that can be included for social sustainability assessment are: health and safety, quality of working conditions, impact on employment, education and training, knowledge management, innovative potential, customer acceptance and societal product benefit, and social dialogue . SLCA provides the critical indicators, metrics to identify the social hotspots.
As Heijungs et al.  pointed out, LCA, LCC and SLCA can be considered as three ways of looking at the same system.
2.4. Call for Life Cycle Sustainability Assessment
To understand the complexity of developing sustainable and resilient systems, new system-based integrated metrics and modelling tools with consideration of triple pillars of sustainability, stakeholders' interests and their interactions should be developed. We need to find the critical ones (i.e., hotspots) for environmental (LCA), social (SLCA) and economic (LCC) dimensions that really dictate the development of sustainable, low carbon, climate change-resilient systems. Pareto principle (also known as the 80–20 rule) or law of the vital few states that roughly 80% of the impacts come from 20% of the main causes . Figure 1 shows the integrated methodological framework for assessing a system crossing temporal and spatial dimensions. Kloffer [57,58] proposed that Life Cycle Sustainability Assessment (LCSA) = LCA + LCC + SLCA given that similar system boundary and functional unit is considered when comparing alternative systems. However, we still need to take into account the interlinkages of the hotspots and consider the multi-stakeholders inputs to create a more robust integrated life cycle sustainability assessment for any system at an open, transparent virtual level.
Computational tools for sustainability assessment can help in modelling the critical elements that affect system behaviour to achieve economic, social and environmental efficient outcomes. The environmental, social and economic hotspots identified using the above life cycle thinking methods are critical elements for system modelling to start with. Here, system modelling and simulation is considered as “laboratory experiment” which abstracts what occurs in the real world by creating a prototype model starting with those identified critical elements [1,14,59,60]. This is also a far less expensive way of studying an engineered real complex system. Together with established tools and models in environmental, social and economic assessments as described above, system dynamics and agent based modelling can be used to account for interconnections and thus create a dynamic computational sustainability assessment for any system investigated.
Obviously, the methods explained above have their own respective data availability challenges. LCA databases are considered well-advanced, though the developments are still on-going for quality datasets and databases. There is also an OpenLCA with latest RECIPE LCIA impact assessment method which is also an on-going development .
We will describe next how to incorporate stakeholders' interests as well as the dynamic system modelling of the interplay of various environmental, social and economic criteria, indicators, metrics and parameters over time.
3. Incorporating Stakeholders Inputs and Dynamic Interconnections
Heijungs et al  insinuate stakeholder involvement, value judgements and interactive multi-criteria based activity in support of developing LCSA. However, they did not specify the use of well-known established methods of multi-criteria decision making. The next aspect that we need to take into account in developing life cycle sustainability assessment is that there are multi-stakeholders in evolving or existing systems. These stakeholders have their own respective environmental, social and economic criteria and interests for developing a sustainable system. Only a few industrial ecologists or sustainability-oriented engineers and scientists are fully aware about the power of multi-criteria decision analysis (MCDA) or multi-objective decision making (MODM). The fields of industrial system engineering, operations research and management science have used these methods to different decisions and policies to account the competing interests of stakeholders and/or experts as well as to produce more transparent, open and robust decisions and policies.
3.1. Multi-criteria Decision Analysis for Stakeholders' Analysis
MCDA is an operational evaluation and decision support system that is suitable for addressing complex problems featuring high uncertainty, conflicting objectives, different forms of data and information, multiple interests and perspectives, and including complex evolving bio-physical and socio-economic problem [61-66]. It has long been widely applied to economic, social, and industrial systems. An MCDA in general involves m alternatives (e.g., bioenergy systems) evaluated on n criteria (i.e., sustainability criteria), in which each of j-th criteria C of i-th alternative A has performance of xij. Each criterion is weighted, and wj is the weight of criteria j. The grouped (i.e., stakeholders) decision matrix X can be expressed as seen on Figure 2.
Wang et al.  present a review of MCDA methods to aid in sustainable energy decision making. In their review, the corresponding methods in different stages of multi-criteria decision-making (i.e., criteria selection, weighting, evaluation and final aggregation) are discussed. The criteria are classified in four major aspects: (i) technical (e.g., efficiency, primary energy ratio, etc.), (ii) economic (e.g., investment cost, net present value, etc.), (iii) environmental (e.g., CO2 emission, NOx emission), and (iv) social (e.g., social acceptability, job creation, etc.). The weighting methods of the criteria are classified into three categories: (i) subjective weighting (e.g., pair-wise comparison, analytical hierarchy process (AHP), etc.), (ii) objective weighting (entropy method, technique for order preference by similarity to ideal solution, etc.), and (iii) combination method.
Different MCDA/MODM approaches have been applied to support different decisions including environmental and sustainable energy decision making [68-70]. We can use any of MCDA methods to aid stakeholders' analysis to find out the “most critical” criteria, indicators and metrics that represent stakeholders' interests. Pineda-Henson et al.  had used the combination of AHP and LCA for evaluating environmental performance of pulp and paper manufacturing. Halog [1,69] proposes the use of analytic hierarchy process (AHP), one of MCDA methods [72-74] in stakeholders' analysis for identifying the critical criteria, indicators and metrics which represent multi-stakeholder's interests. This will provide ranking of different criteria, indicators and metrics which are important holistically. MCDA opens great applicability to support sustainability assessment of existing and emerging multi-attribute systems. For instance, AHP allows stakeholders to weigh the different criteria, indicators, and metrics by calculating Eigen values. In biofuels system, where energy efficiency, investment cost, GHG emissions, land use change and social impacts are the most common criteria, MCDA is certainly applicable. Through MCDA, we can focus on the critical ones that account stakeholders' inputs. The idea follows Pareto Principle and the principle of factor sparsity which state that roughly 80% of the impacts come from 20% of the causes . This means that we should focus on the critical criteria and variables that dictate the sustainability performance of any system.
By identifying the hotspots in LCA, LCC and SLCA as well as the important criteria from stakeholders' perspectives, we can reconcile these hotspots and criteria to eliminate more or less similar indicators and metrics; and to produce the vital few variables and metrics which are critical for developing sustainable and resilient systems. Through these complementary approaches, we can account stakeholders' voices as well as capitalize the strengths of life cycle thinking methods. If we still doubt about the veracity of results, we can perform uncertainty and variability analyses (e.g., perturbation analysis) to convince ourselves that we selected the critical ones.
However, the existing life cycle thinking and MCDA methods are considered steady-state methods whereby they provide snapshots of hotspots based on historical data. They do not provide projections or trends in the future. They do not take into account the interactions of different metrics, outputs and parameters over time. To make the results more useful for decision and policy makers, we need to model the dynamic interrelationships of these variables over time. Additionally, we can explore the use of geographic information systems (GIS) to assist spatial analysis if needed.
Before discussing the two complex systems (non-linear) modelling methods that we suggest here—agent based and system dynamics, let's highlight first the usefulness of data envelopment analysis to calculate eco-efficiency metrics and other composite indicators which will enrich our system modelling efforts as well as useful for corporate decision makers. This is in line with Heijungs et al.  suggestion to calculate combined indicators such as eco-efficiency for broader LCA.
3.2. Data Envelopment Analysis for Eco-Efficiency and other Ratio Calculations
If we want to calculate eco-efficiency indicators (ratio of environmental to economic information) or any other ratios (e.g., material/energy intensity; resource productivity), data envelopment analysis (DEA) method can be used. One of the few eco-efficiency metrics that attempted to integrate both environmental and economic performance was developed by Unilever [75-78], which is known as Overall Business Impact Assessment (OBIA). DEA is a data-driven and based on mathematical programming which can aid to identify which critical factors need to be addressed for improving the performance of decision making units (DMU) towards achieving efficiency . DEA is an operations research/management science based empirical procedure for estimating the relative efficiency of any decision making units (e.g., technologies, suppliers, manufacturers, supply chains) [80-82]. Tyteca [83,84] and Kuosmanen and Kortelainen [85,86] developed different and complex variations of DEA models. The definition of a DMU can vary greatly, as long as the units or agents or systems can be modelled either as inputs or outputs. The inputs and outputs in DEA models may have different units of measurement and they can be under certain assumptions even qualitative [87,88].
A DEA-based OBIA for assessing the eco-efficiency performance of industrial economic sectors in Japan was proposed by Halog et al. . The advantage of DEA-based efficiency models is that the causes of inefficiency can be identified and the extent of reducing resource consumptions and emissions, increasing the economic value added, or improving social benefits (e.g., increase employment) can be relatively quantified . Figure 3 shows a sample output of DEA analysis which is reported in Halog  study and where he proposed models for evaluating energy, environmental and sustainability performance of biofuels value chains.
Making any systems eco-efficient is a necessary step towards sustainability [90,91]. We can start from investigating a set of necessary conditions (i.e., backcasting) that a system must fulfil in order to become sustainable. Necessary conditions can include being efficient in the use of energy and other resources, in the reduction of environmental emissions, in the social role played by system's actors as reflected by their rates of employment, the working conditions, and in the care taken with respect to future generations in the setting of long-term objectives. Halog  described sustainable network operational efficiency (SNOE) which refers to the overall efficiency of all activities included in a supply chain network. The efficiency can be improved by reducing or increasing chain's complexity, reducing the cost of doing business with other activities in the chain (which may result to increase economic value), increasing societal benefits as well as reducing environmental emissions and resource consumptions. The cumulative effect of improving these factors of network operations is to increase its overall sustainable value. For example, if a chain of operations whose efficiency is determined to be relatively poor compared to others; then, as a whole, this inefficient network is expected to have low overall value added, high environmental emissions, high energy consumption, long lead-time to order products and to deliver services, and so on.
DEA allows for simultaneous analysis of multiple inputs and multiple outputs. In productive efficiency perspective, input-output (I/O) analysis quantifies inter-organizational relationships and offers a means of evaluating the impacts across firms. Similar thinking can be applied for the analysis of social impacts of operations (e.g., direct and indirect employment) within a network.
3.3. Agent Based Modelling (ABM)
ABM model is a computer representation of the considered system that is comprised of multiple, interacting actors (i.e., agents) . ABM systems possess two distinct properties: (1) the system is composed of interacting agents; and (2) the system exhibits emergent properties, that is, properties arising from the interaction of the agents that cannot be deduced simply by aggregating the properties of the agents [93,94]. ABM can be used to model the interactions of agents or sub-systems using the metrics, variables and indicators as performance measures. Figure 4 provides schematic illustration of an agent-based system: each of the four circles represents a sub-system of agents (e.g., companies) denoted by small dots and the whole arrows show how agents and sub-system of agents are interacting with each other. Interacting agents and sub-systems, though driven by only a small set of rules which govern their behaviour, account for complex system behaviour whose emergent dynamic properties cannot be explained by analyzing its component parts . Therefore, ABM aims to look at global consequences of individual or local interactions in a given geographical area.
The critical attributes of agents can be expressed in continuous measurements or discrete categories. The actions of agents can be scheduled to take place synchronously or asynchronously. The behaviours of agents can vary from completely reactive, i.e., agents only perform actions when triggered to do so by some external stimulus (e.g., actions of another agent), to goal-oriented (e.g., through seeking a particular goal).
Parker et al.  have categorized existing literatures on agent-based land use models into five categories—(i) policy analysis and planning; (ii) participatory modelling; (iii) explaining spatial patterns of land use or settlement; (iv) testing social science concepts; and (v) explaining land use functions. Different business and technical problems have been modelled using ABM since 1990s. Recent examples of ABM applications range from supply chain optimization to consumer behaviour, from analyzing growth and decline of ancient civilization to decision support system for traffic and transportation system [96,97]. Fox et al.  argue that optimization of supply chain performance is only possible when the impacts of decisions made by one unit onto another units are understood. A systems model that captures all important interactions among different units of a supply chain would contribute to effective decision making . Swaminathan, Smith, and Sadeh  demonstrate the use of object-oriented agent modelling in more detail to increase overall performance of a supply chain. Julka et al.  use petroleum refinery integrated supply chain modeller and simulator to emulate a crude refinery's supply chain to develop procurement strategies. ABM has been used in ecology too. Athanasiadis et al.  present a conceptual model for building Environment Information System (EIS) using two types of agents. They argue that a model like this with a large volume of EIS databases can be simulated to develop scenarios of future environmental conditions given the importance of justifying environmental benefits of any projects. For instance, ABM enables users to analyze how sensitive is the biofuel supply chain to the change of production factor (i.e., yield), or how severe is the environmental impact caused by shifting one technology to another. ABM could conveniently model the complex behaviour of system participants (such as autonomy, intelligence, and reactions); this is relevant in biofuel system because of the complexity of the system itself and the dynamics behaviour of the system's entity, such as market participants and social stakeholders.
Several agent-based simulation (ABS) packages, which can be used to develop ABS models for various kinds of systems are available. In general, ABS packages can be divided into two types: toolkits and software . Toolkits provide libraries with some specific functions designed for ABS. Most of the toolkits have been developed by academic research centres/groups and contain state-of-the-art features. Popular ABS toolkits include SWARM, Repast, MASON and NetLogo [103-106]. There is also the OpenABM at www.openABM.org which contains a collection of tutorials on computational modelling and a modelling library where authors and modellers can share their models.
3.4. System Dynamics (SD)
SD is a well-established systems perspective/complexity science method which is originally developed by Jay Forrester at MIT [107,108]. This has been applied in different corporate, industrial and government decisions worldwide which have the intention of understanding and modelling the interrelationships (i.e., feedbacks) of variables, indicators and metrics over time. This is also useful in modelling the interrelationships between or among sub-systems which are linked by metrics or variables and aids to see how their interrelationships will produce specific overall system behaviour. Before using appropriate modelling software package, it is important to draw causal loop diagrams. A causal loop diagram is a visual representation of the feedback loops in a system whereby the stocks and flows (i.e., involving different variables, parameters, indicators and metrics) are connected by either positive and negative loops. A stock (e.g., biomass, GHG, unemployment) is the term for any entity in the system that accumulates or depletes over time. A flow is the rate of change in a stock. A flow changes the rate of accumulation of the stock. The real power of system dynamics is utilized through simulation and in showing diagrams that model the interlinkages between micro-, meso-, and macro-systems. SD involves computer simulation modelling for framing, understanding, and discussing complex issues and problems [109-111]. It is now recognized that the structure of any emerging system—the many circular, interlocking, sometimes time-delayed relationships among its components—is often just as important in determining its behaviour as the individual components themselves. As insinuated in the introduction section, there are often properties of the whole which cannot be found among the properties-of-the-elements. The feedback loops as well as the use of stocks and flows can represent and model the critical sustainability variables, indicators, and metrics to describe how seemingly simple sub-systems display baffling nonlinearity for the whole system . The modelling can be developed by sub-dividing the whole system into sub-models but bearing in mind that they are interconnected by variable, parameter or a metric [14,46]. Through SD we can create a prototype dynamic system model for the system being considered.
In SD, a system is modelled mathematically in a nonlinear, first-order differential (or integral) equation such as:
A suitable computer software packages are also available to facilitate the development and simulation of an SD model of a system being investigated. STELLA, Powersim and Vensim are well-known software packages for SD computation. SD typically goes further and utilizes simulation to study the behaviour of systems and the impacts of alternative policies [60,110,111]. Running “what if” simulations or scenarios to test certain energy and environmental policies on a prototype system model can greatly aid in understanding how an emerging system potentially evolves over time. Similar to MCDA and ABM methods, SD has been applied in a wide range of areas, for example population, ecological and economic systems, which usually interact with each other. SD has been used in the sustainability assessment of technologies in the Canadian Oil Sands Industry [14,46] as well as in bioethanol production in Canada . SD models have recently been developed in some biofuels studies. Riley et al.  use SD model to describe the U.S. Department of Energy biomass program. Bush et al.  and Sheehan  explore the potential market penetration scenarios for bio-fuels in the United States. Scheffran and BenDor  investigate interaction between economic conditions and land competition between different crops. Franco et al.  use SD to understand the difficulties in fulfilling government requirements for biofuels blending and to evaluate the effect of different government policies in the production of ethanol and biodiesel. The value of dynamic system modelling is tying together micro-, meso-, and macro-level impacts (which have been identified in LCA, LCC, SLCA, MCDA, DEA methods) in implementing any complex system over time.
Once a prototype system model is validated, the procedural steps can be done repeatedly depending on the scenario considered. This will also provide information about the trends of important variables over time which will give us insights and guidance on what decisions and policies to take. Eventually, this modelling procedure can support the selection and implementation of sustainable systems. Thus, application of SD and ABM as platforms in which other methodologies, such as LCA, MCDA and DEA can be integrated, will open an opportunity for a more holistic dynamic sustainability assessment.
The two dynamic system methods above can work synergistically with Sustainable Network Theory (SNT) proposed by Kim et al.  which is discussed below. Again, SNT, ABM and SD methods are not competing but they are complementary for the purpose of enriching the results and supporting for optimal, sustainable decision and policy making regardless of the scale of system considered. The integrated framework can be used (with the use of system dynamics in particular) to analyze the scaling up of any emerging technology such as biofuel technologies. Taking into account environmental, economic and social dynamics within a geographical region, ABM and SD can build appropriate model necessary for a more robust dynamic sustainability evaluation. In addition to that, available ABM and SD packages can help user to develop dashboards for better interpretation and visualization of results. Finkbeiner et al.  showed the use of dashboards to present life cycle sustainability indicator results to different stakeholders. This formatting of results can also be done using the latest software packages for LCA, system dynamics and agent based modelling or simply using MS Excel package.
3.5. Sustainable Network Theory (SNT)
Though SNT can overlap with SD, it is important to note that these methods are complementary to each other. SNT models the understanding and visualization of a complex system by conceptualizing it as a network. A conceptual SNT framework made up of a social, cultural and regulatory network; an economic and financial network; and a material, energy and environmental flow network is shown in Figure 5. The theories of networks and network analysis have been widely applied providing a unifying language to describe disparate systems ranging from social interactions to power-grids [118-120]. Kim  showed an SNT framework for industrial systems which are understood in a holistic sense to include social and industrial dimensions, manufacturing and service components, and consumption and end-of-life activities. Thus, the science of networks is a promising vehicle by which to study, and advance our understanding of complex systems that are at the heart of sustainability science and engineering. Network theory and analysis can support identification of causal loops (in the case of system dynamics), help with prioritization of conflicting factors (such as in MCDA), and facilitate policy intervention and implementation at the right levels.
3.6. Scenario Development and Analysis for Policy Planning and Making
Forecasting, foresighting and backcasting are approaches used for policy planning and making. These scenario development approaches have their benefits and shortfalls. Backcasting involves working backwards from a particular desired future end-point or set of goals (i.e., sustainable society) to the present state, in order to determine the physical feasibility of that future and the policy measures that would be required to reach the state [121-123]. This helps in analyzing alternative futures responsive to present situation and deals with problems in a different way rather than extrapolating present scenario into the future (forecasting) [124-127]. Backcasting makes it clear that addressing sustainability concerns requires a paradigm shift from business-as-usual attitudes. On the other hand, industries and organizations use forecasting technique as a data analysis methodology to develop future scenarios from existing information. Forecasting enables decision makers to identify reasonable estimates of various current activities. Using forecasting approach, managers and decision makers can tweak and calibrate their operations at the appropriate time in order to maximize benefits. Forecasting assists in preventing losses by taking in all relevant information and making proper judgment decisions . Moreover, foresighting can be distinguished from forecasting. Forecasting is the passive attempt to diagnose or predict future events. Foresighting aims to actively change or create the future by linking it to the present. Thus, the major difference between foresight and forecasting is that in forecasting the conclusions for today are missing. There are four major applications of foresighting: (1) assessing possible consequences of actions, (2) anticipating problems before they occur, (3) considering the present implications of possible future events; and (4) envisioning desired aspects of future societies. Foresighting as a tool for ‘decision-shaping’ rather than ‘decision-making’ offers many benefits including: engaging policy-makers and experts in actively planning for the future, identifying potential problems early, verifying expectations and examining trends, bringing people together to create a suitable future, strengthening existing networks, and educating the public on urgent future-related issues. It could have a positive impact on sustainable technology policy by providing a means for analyzing its broader social and economic implications. For instance, while some believe that nanotechnology has the potential to eliminate the problem of resource scarcity, others have pointed out that a technology which allows that ‘anything can be made from anything’ is sure to have an impact on our ecological systems.
By considering different scenarios (starting from business-as-usual approach to different plausible scenarios in the future (either through foresighting or backcasting), we can generate different results for our system performance measures and identify a few critical alternative systems or scenarios to be strongly considered for developing sustainable decision, policy, technology, systems, intervention, etc. Again, this is with the assumption that we can gather good quality data. We can also perform sensitivity and uncertainty analyses to improve the robustness of our results. There are lots of statistical tools, qualitative and quantitative analysis suggested by LCA, and modelling and simulation textbooks to strengthen the accuracy of our results.
Through integrated systems modelling and assessment, information moves beyond pure science and becomes both salient and legitimate to decision and policy making processes. The inclusion of stakeholder perspectives and including the dynamic interactions of different variables in the proposed framework ensures relevance—another criterion for sound life cycle sustainability assessment.
4. Ongoing Applications of Integrated Sustainability Assessment Framework
Regardless of the scale of the system considered—micro, meso and macro systems, we can apply the proposed integrated system methodological framework above for dynamic life cycle sustainability assessment. The implementation of this integrated approach relies heavily on available quality databases. The creation of interconnections between and among databases is possible with the use of information and communication technologies (i.e., web-based databases, cloud computing). Computational Sustainability advocates the creation of virtual, open well-connected databases where data can be mined for the purpose of making sustainable decisions and policies in government, industries and corporations. This really creates a platform for win-win situation or more collaborations than competitions among the players. To pursue collaborative projects, we also need to develop mutual trust and put the betterment of the whole global society above our respective self-interests.
The integrated methodological framework is currently being applied to the following case studies, though we will focus our discussion here on our initial work in developing sustainable biofuels supply chains.
Development of a sustainable supply chain for wood-based bioethanol;
Development of a sustainable supply chain for palm-oil based biodiesel in Indonesia; and
Development of a low carbon, low resource eco-industrial park in Orono/Old Town, Maine where pulp and paper industry is the core industry.
4.1. Sustainable Supply Chain of Biofuel Systems
Renewed interest in bioenergy development can be attributed to the declining stock of fossil fuels, energy independence, environmental benefits and renewability characteristic of biomass feedstocks. Furthermore, the ambitious target set by the U.S. to produce 30 percent of transportation fuels to come from renewable sources by 2030 insinuates the urgency to explore sustainable avenues of biomass procurement/production, efficient conversion technologies, transportation networks and end-uses. The Billion Ton Report prepared by Oakridge National Laboratory (ORNL), on behalf of US Department of Agriculture (USDA) and US Department of Energy (USDOE), identified a biomass potential of 1.3 billion dry tons per year from forestland and agricultural land in the US . The study reports that the biomass could meet more than one-third of the nation's transportation fuel demand. In the context of Maine, Dickerson et al.  conducted a similar study to calculate biomass availability but adjusted to the more up-to-date forest inventory data. Their study concludes that Maine has a biomass potential of 2.6 million dry tons of forest residues and 8.3 million dry tons of round wood products per year which could be harvested sustainably. Combined together, the total available biomass can produce about 833 million gallons of ethanol using hydrolysis and fermentation processes, which is equivalent to about 77 percent of gasoline consumed in Maine in 2005.
Forest Bioproducts Research Institute (FBRI) at the University of Maine has already started works on technology assessment for forest bio-refineries. The databases in Maine Forest Service, USDA, USDOE/NREL, etc. have been used, however these data should be refined and analyzed further for specific requirements of the study. Though numerous data required for the study can be used from existing databases and peer-reviewed literatures, some specific data required are collected through lab/pilot scale data, field works, interviews and even questionnaire surveys.
Figure 6 shows a cradle-to-grave supply chain of biofuels while considering the impacts representing the triple dimensions of sustainability. Social and ecological impacts are the direct inevitable consequences of biofuels system development. The current study has adopted the key criteria (which are converted into relevant variables and parameters) identified in the works of Buchholz et al. [130,131] on sustainability criteria for bioenergy systems. The study analyzed how key bioenergy stakeholders and experts perceived the list of 35 initially identified sustainability criteria for bioenergy (as shown in Table 1) found in emerging sustainability assessment frameworks . Using AHP, participants were asked to rate each of the 35 criteria on four attributes including relevance, practicality, reliability, and importance. Similar to Buchholz et al  and Elghali et al. , through stakeholders' analysis of bioenergy production, the range of stakeholders involved are broad and identified in Figure 7.
4.2. Dynamic System Modelling for Sustainability Assessment of Forest Biofuels Supply Chain
Based on the selected criteria identified in AHP results, a preliminary causal loop diagram (as shown in Figure 8) of bioenergy production demonstrates how the different sectors add up or resolve the intricacies associated in the biofuels supply chain . Even when biomass use for biofuel production becomes more lucrative, it takes some time for the industry to respond to such impulses. System dynamics capture these kinds of impacts that occur only after some time interval. It will help to understand the interrelationships among different components of a supply chain and thus provides insights on systems synergy. In Figure 8, biomass contributes positively to biofuels production. Production factor (technological maturity) enhances the production process generating more revenue. At the same time, emissions caused by the production process accumulate. Policies often intervene to cut-off emission levels. Economic allocation determines the revenue share for biomass production, emission control and reserve (liquid assets holding). A higher level of emissions in the cycle contributes to environmental degradation. In the case of the bio-industry, biodiversity loss, water shortage and resource degradation will potentially increase. The environmental degradation would have social, economic, institutional and obviously environmental implications. Ultimately, the biomass from where the process began feels the pressure. The causal loop diagram shows the direct and indirect relationships among different sectors. The direct relationships and their impacts are largely understood and are addressed through policies. As indicated by Figure 8, some units of the biofuel supply chain system take a long time to react to the changes in other units when they are not directly linked to each other. Many of the processes involved are extremely complicated and it may take several decades to witness significant changes in their behaviours. This is particularly true in context of ecosystem functioning as the causal relationships are non-linear and have inertial characteristics .
The relationships between different sectors or agents in biofuels supply chain are subject to social, economic and environmental limitations. The limitations imposed in terms of biomass production, revenue generated by companies engaged in production and the environmental emissions associated with production and uses will all change when one of the sub-systems equilibrium is disturbed. A dynamic system model of the bioenergy production is being built using system dynamics modelling software. Using the framework (as shown in Figure 1), we will understand the system behaviour at individual/agent level and in an integrated level. This will help us identify causes and effects of different consequences (e.g., interrelationships and feedbacks) that evolve over time in forest bioenergy industry.
4.3. Model Validation and Characterization of Uncertainties
Baumgartner  argued that assessing environmental and social impacts is associated with uncertainties caused by applied assessment tools, definition of assessment objectives, system boundaries of assessment and data quality. There are various ways to deal with data and model uncertainties when conducting system modelling and simulation [15,135]. Uncertainties and variabilities come from a large number of variables and parameters considered, assumptions made, and the spatial and temporal variability in parameters or sources [97,136,137]. The latest LCA, MCDA, and system dynamics software packages include statistical tools to support uncertainty and sensitivity analyses. Uncertainty analysis aids to show if the model's general pattern of behaviour is strongly influenced by changes in critical parameters. For system dynamics, the usual method is to perform a sensitivity analysis of the model whereby a collection of simulated experiments is performed . This is done by choosing parameters, metrics and indicators that are judged to be sensitive, changing their values and then re-running the simulation model. If there is a drastic response in the results, this can show a lack of robustness in the system model. For ABM, the goal is to check whether the model addresses the right problem and provides accurate information about the system being modelled . Additionally, Miller  proposes to use computer-based Active Nonlinear Tests (ANTs) that are capable of performing multivariate sensitivity analysis, model breaking and validation, extreme cases, and policy discovery. ANTs search across sets of parameter values and are capable of detecting important non-linear relationships among the parameters—relationships that typically go unnoticed using standard techniques .
5. Conclusions and Suggestions for Future Works
In pursuit of addressing urgent sustainable development issues, the integrated systems modelling framework proposed here is a big attempt to connect the basic principles and disparate methods of science, engineering and technology to formulating sustainability-related government policies and corporate decisions. The integrated framework seeks to enable the discoveries needed to inform actions that lead to environmental, energy and societal sustainability. This study provides the conceptual, theoretical, empirical, and computational background needed to further develop the basic science, engineering, education, and policy knowledge base relevant to sustainability. The framework will also support projects at multiple scales, from the individual to the system level, and will stimulate innovations in education and learning research and practice. The expected results of applying the framework can be used to facilitate effective inter-organizational strategic decisions and government policy planning and making.
The integrated methodological framework is currently being applied to development of supply chain networks of wood-based ethanol and palm-oil biodiesel as well as creation of ecologically sustainable forest-based eco-industrial park in Maine. Through integrated system modelling, we can develop a computer-based sustainable system model that follows the principles of industrial ecology, ecological, evolutionary and biophysical economics. The purpose of systems modelling and simulation with the use of agent-based modelling and/or system dynamics is to generate, explore alternative sustainable futures that may develop under different conditions. ABM (inductive-approach) is appropriate when we look at global consequences of individual or local interactions in a given space such as modelling the participation of agents in emerging biofuels supply chain while SD (deductive-approach) is useful for tracing the patterns of behaviour of a dynamic supply chain system to its feedback structure. Both SD and ABM have been applied to an equally wide range of fields such as biology, ecology, economics, education, engineering, medicine, public administration and policy and design. By combining them, the hybrid can provide further insights in comparing the aggregate behaviour and emergent influence on the environment of agent-based models with the predictions of aggregate-level feedback models in engineered complex systems. The integrated system model developed can aid in exploring various “what if” scenarios considering the economic, social and environmental goals of a particular system considered. The models can show the possible evolutionary trajectories of given scenarios under different conditions and geographical contexts over time. The system model aids to foresee potential outcomes, and may lead to improvement in managing ecological and engineered complex systems. The integrative approach is capable to assess responses when there are changes in system behaviour. Such changes may result from extreme and unexpected events, where systems are stressed beyond usual patterns of behaviour, or from small changes in social phenomena, such as changes in energy use, land use, or the introduction of new actors or economic incentives, etc.
Moreover, the integrated framework is expected to be useful for addressing challenges in climate and energy research and education to understanding, predicting, and reacting to change in the linked natural, social, and built environment. The framework can be explored to support the achievement of the following endeavours:
research at the energy-environment-society nexus;
novel energy production technologies that minimize environmental impact and corresponding adoption, socioeconomic, and policy issues;
innovative computational science and engineering methods and systems for monitoring, understanding and optimizing life-cycle energy costs and carbon footprints of natural, social and built systems;
data analysis, modelling, simulation, visualization, and intelligent decision-making facilitated by advanced computation to understand impacts of climate change and to analyze mitigation strategies;
support for interdisciplinary education/learning science research, development, and professional capacity-building related to sustainability science and engineering;
development of the workforce required to understand the complexities of environmental, energy, and societal sustainability;
engaging the public to understand issues in sustainability and energy; and
development of cyberinfrastructure needed to enable sustainability science and engineering.
Though we are using different software packages and modelling tools to implement the framework, we strongly recommend that the complementary strengths of the system methods used in the framework should be integrated computationally through the use information and communication technologies such as cyberinfrastructure, cloud computing, artificial intelligence, etc. Cyberinfrastructure supports advanced data acquisition, storage, management, integration, mining and visualization as well as other computing and information processing services over the Internet [10,12]. It is considered a technological solution to the problem of efficiently connecting data, software packages, computers, and people with the goal of enabling derivation of novel scientific theories and knowledge (i.e., development of computational-based dynamic life cycle sustainability assessment of any systems). Cloud computing is Internet-based computing, whereby shared servers provide resources, software, and data to computers and other devices on demand. Through cyberinfrastructure, we can advance the development of good quality and well-connected databases to support transformative, collaborative, cross-disciplinary and transdisciplinary research for integrated sustainability assessment.
Finally, we envisage that the proposed integrated system modelling method with the enabling support of cyberinfrastructure will become the basis for computational based life cycle sustainability assessment of systems at various organizational levels. The integrated framework can be used to advance the role of science in transitioning toward sustainability, building upon the conceptual distinction between problem-solving and critical research strategies as starting point. We would like to encourage the academic, government and business communities to collaborate in using the framework described here and apply to respective systems that they deal with. Through integrated computational approach for sustainability assessment, industrial ecology-oriented approach for human and industrial development can potentially lead us to a sustainable co-existence of economy, technology and natural environment. Thus, the framework provides new ways of integrating knowledge across the divides between social and natural sciences as well as between critical and problem-solving research.
|Table 1. 35 Identified Sustainability Criteria for Biofuel Systems Using Stakeholders/Experts Analysis. Adapted from .|
|Criterion name||Nature of criterion||Relevance Rating||Practicality Rating||Reliability rating||Importance rating|
|Greenhouse gas balance||Environmental||2.84||2.33||2.17||3.55|
|Natural resource efficiency||Environmental||2.78||2.02||1.86||3.11|
|Compliance with laws||Social||2.46||2.13||1.95||3.09|
|Monitoring of criteria performance||Social||2.73||2.12||2.02||3.02|
|Adaptation capacity to env. hazard||Environmental||2.63||2.05||1.80||2.90|
|Working condition of workers||Social||2.65||2.27||1.98||2.83|
|Use of chemicals, fertilizer, pest control||Environmental||2.53||2.23||2.07||2.72|
|Potentially hazardous atmospheric emission other than greenhouse gases||Environmental||2.57||2.26||2.17||2.72|
|Property rights and right||Social||2.55||2.00||1.76||2.68|
|Land use change||Environmental||2.40||1.79||1.64||2.68|
|Use of genetically modified organisms||Environmental||2.44||2.07||1.85||2.64|
|Respect for human rights||Social||2.28||1.55||1.50||2.48|
|Exotic species applications||Environmental||2.18||1.88||1.69||2.33|
|Land availability for other human activities than food production||Social||2.18||1.70||1.63||2.25|
|Standard of living||Social||2.14||1.77||1.67||2.14|
|Overall average rating||2.49||2.01||1.87||2.75|
|Consensus (std. deviation)||0.25||0.26||0.25||0.38|
We are grateful to some financial support from the Maine Agricultural and Forest Experiment Station (MAFES) with a Publication Number of 3167. We also acknowledge the efforts of the reviewers.
- Halog, A. Sustainable development of bioenergy sector: An integrated methodological framework. Int. J. Multicriteria Decis. Making 2011. in press. [Google Scholar]
- Heijungs, R.; Huppes, G.; Guinee, J. Life cycle assessment and sustainability analysis of products, materials and technologies: Toward a scientific framework for sustainability life cycle analysis. Polym. Degrad. Stabil. 2010, 95, 422–428. [Google Scholar]
- Heijungs, R.; Huppes, G.; Guinee, J.B. A scientific framework for LCA; Deliverable 15 of the CALCAS project. 2009. Available online: http://www.estis.net/sites/calcas/ (accessed on 16 February 2011). [Google Scholar]
- Rotmans, J. Tools for integrated sustainability assessment: A two-track approach. 2008. Available online: http://academic.research.microsoft.com/Paper/3999336.aspx (accessed on 16 February 2011). [Google Scholar]
- Elghali, L.; Clift, R.; Sinclair, P.; Panoutsou, C.; Bauen, A. Developing a sustainability framework for the assessment of bioenergy systems. Energ. Policy 2007, 35, 6075–6083. [Google Scholar]
- Robert, K. Tools and concepts for sustainable development, how do they relate to a general framework for sustainable development, and to each other? J. Clean. Prod. 2000, 8, 243–254. [Google Scholar]
- Bertrand, N.; Jones, L.; Hasler, B.; Osmodei-Zorini, L.; Petit, S.; Contini, C. Limits and targets for a regional sustainability assessment: An interdisciplinary exploration of the threshold concept. In Sustainability Impact Assessment of Land Use Changes; Helming, K., Ed.; Springer: Berlin, Germany, 2008; pp. 405–424. [Google Scholar]
- Hacking, T.; Guthrie, P. A framework for clarifying the meaning of triple bottom line, integrated, and sustainability assessment. Environ. Impact Assess. Rev. 2008, 28, 73–89. [Google Scholar]
- Allenby, B. Creating economic, social and environmental value: An information infrastructure perspective. Int. J. Environ. Tech. Manag. 2007, 7, 618–631. [Google Scholar]
- American Council of Learned Societies (ACLS). Report on Cyberinfrastructure. 2006. Available online: http://www.acls.org/programs/Default.aspx?id=644 (accessed on 16 February 2011).
- Gomes, C.; Sabharwal, A.; Selman, B. Using Cloud Computing for Computational Sustainability, 2nd ed.; Open Cirrus Summit: Sunnyvale, CA, USA, 29 January 2010. [Google Scholar]
- National Science Foundation (NSF). Cyberinfrastructure vision for the 21st century. March 2007. Available online: http://www.nsf.gov/pubs/2007/nsf0728/nsf0728.pdf (accessed on 16 February 2011). [Google Scholar]
- Guinee, J.; Heijungs, R.; Huppes, G.; Zamagni, A.; Masoni, P.; Buonamici, R.; Ekvall, T.; Rydberg, T. Life cycle assessment: Past, present, and future. Environ. Sci. Technol. 2010, 45, 90–96. [Google Scholar]
- Halog, A.; Chan, A. Developing a dynamic systems model for sustainable development of the Canadian oil sands industry. Int. J. Environ. Technol. Manage. 2008, 8, 3–22. [Google Scholar]
- Halog, A. Models for evaluating energy, environmental and sustainability performance of biofuels value chain. Int. J. Global Energy Issue. 2009, 32, 87–101. [Google Scholar]
- Gasparatos, A.; El-Haram, M.; Horner, M. A critical review of reductionist approaches for assessing the progress towards sustainability. Environ. Impact Assess. Rev. 2008, 28, 286–311. [Google Scholar]
- Allenby, B. Teaching and educational programs in sustainability. Proceedings of 2nd International Congress on Sustainability Science and Engineering (ICOSSE), Tuczon, AZ, USA, 9–12 January 2011.
- System Dynamics Society. Available online: http://www.systemdynamics.org/ (accessed on 16 February 2011).
- Hawken, P. The Ecology of Commerce: A Declaration of Sustainability; Harper Business: New York, NY, USA, 1993. [Google Scholar]
- Johnson, P.; Everard, M.; Santillo, D.; Robert, K. Reclaiming the definition of sustainability. Env. Sci. Pollut. Res. 2007, 14, 60–66. [Google Scholar]
- O'Riordan, Timothy. Ecological Economics: The Science And Management Of Sustainability; Costanza, R., Ed.; Columbia Univ. Press: New York, NY, USA, 1991. [Google Scholar]
- Chiu, Y-W.; Walseth, B.; Suh, S. Water embodied in bioethanol in United States. Environ. Sci. Technol. 2009, 2688–2692. [Google Scholar]
- Finnveden, G.; Hauschild, M.Z.; Ekvall, T.; Guinée, J.B.; Heijungs, R.; Hellweg, S.; Koehler, A.; Pennington, D.; Suh, S. Recent developments in life cycle assessment. J. Environ. Manage. 2009, 91, 1–21. [Google Scholar]
- Guinée, J.B. Handbook on Life Cycle Assessment: Operational Guide to the ISO Standards; Kluwer Academic Publishers: Dordrecht, The Netherlands; Boston, MA, USA; London, UK, 2002. [Google Scholar]
- Guinée, J.B.; De Haes, U.; Huppes, G. Quantitative life cycle assessment of products: 1: Goal definition and inventory. J. Clean. Prod. 1993, 1, 3–13. [Google Scholar]
- 14040 International Standards in Environmental Management-Life Cycle Assessment—Principles and Framework; International Organization for Standardization: Geneva, Switzerland, 2006.
- Heijungs, R.; Suh, S. The Computational Structure of Life Cycle Assessment; Kluwer Academic Publishers: Dordrecht, The Netherlands, 2002. [Google Scholar]
- Hendrickson, C.; Lave, L.; Scott Matthews, H. Environmental Life Cycle Assessment of Goods and Services: An Input-Output Approach; RFF Press: Washington, DC, USA, April 2006. [Google Scholar]
- Hawkins, T.; Hendrickson, C.; Higgins, Matthews, H.S.; Suh, S. A mixed-unit input-output model for environmental life-cycle assessment and material flow analysis. Environ. Sci. Technol. 2007, 41, 1024–1031. [Google Scholar]
- Xu, M. Development of the physical input monetary output model for understanding material flows within ecological-economic systems. J. Resour. Ecol. 2010, 1, 123–134. [Google Scholar]
- Zhang, Y.; Singh, S.; Bakshi, B. Accounting for ecosystem services in life cycle assessment, Part I: A critical review. Environ. Sci. Technol. 2010, 44, 2232–2242. [Google Scholar]
- Zhang, Y.; Baral, A.; Bakshi, B. Accounting for ecosystem services in life cycle assessment, Part II: Toward an ecologically based LCA. Environ. Sci. Technol. 2010, 44, 2624–2631. [Google Scholar]
- Ukidwe, N.; Bakshi, B. Flow of natural versus economic capital in industrial supply networks and its implications to sustainability. Environ. Sci. Technol. 2005, 39, 9759–9769. [Google Scholar]
- Udo de Haes, H.A.; Van der Voet, E. Material flow accounting and life cycle assessment. In The Global Environment: Science, Technology and Management; Brune, D., Chapman, D.V., Gwynne, M.D., Pacyna, J.M., Eds.; Scandinavian Science: Oslo, Norway, 1997; Volume 2, pp. 1151–1171. [Google Scholar]
- Brunner, P.; Rechberger, H. Practical Handbook of Material Flow Analysis; Taylor & Francis/CRC: London, UK, 2003. [Google Scholar]
- Fujie, K.; Goto, N.; Kamahara, H.; Hasanudin, U. Materials and energy flow analyses in bio-product processing of plantation. Proceedings of International Seminar on Sustainable Production and Utilization: Challenges and Opportunities, Lampung, Indonesia, 4–6 August 2009.
- Udo de Haes, H.A.; Van der Voet, E.; Kleijn, R. From quality to quantity: Substance Flow Analysis (SFA), an analytical tool for integrated chain management. In Regional and National Material Flow Accounting: From Paradigm to Practice of Sustainability; Bringezu, S., Fischer-Kowalski, M., Kleijin, R., Palm, V., Eds.; Wuppertal Institute for Climate, Environment and Energy: Wuppertal, Germany, 1997; Volume 4, pp. 32–43. [Google Scholar]
- Van der Voet, E.; Van Oers, L.; Guinee, J.B.; De Haes, H.A.U. Using SFA indicators to support environmental policy. ESPR 1998, 6, 49–58. [Google Scholar]
- Bouman, M.; Heijungs, R.; van der Voet, E.; van der Bergh, J.; Huppes, G. Material flows and economic models. An analytical comparison of SFA, LCA and equilibrium models. Ecol. Econ. 2000, 32, 195–216. [Google Scholar]
- Van der Voet, E. Substance flow analysis methodology. In A Handbook of Industrial Ecology; Ayres, R.U., Ayres, L.W., Eds.; Edward Elgar Publishing: Cheltenham, UK, 2002; pp. 91–101. [Google Scholar]
- Pleanjai, S.; Gheewala, S.H. Full chain energy analysis of biodiesel production from palm oil in Thailand. Appl. Energ. 2009, 86, 209–214. [Google Scholar]
- Heinzle, E.; Biwer, A.; Cooney, C. Development of sustainable bioprocesses: Modelling and assessment. In Development of Sustainable Bioprocesses: Modelling and Assessment; John Wiley & Sons, Ltd: Chichester, UK, 2006; pp. 82–113. [Google Scholar]
- Finkbeiner, M.; Schau, E.; Lehmann, A.; Traverso, M. Towards life cycle sustainability assessment. Sustainability 2010, 2, 3309–3322. [Google Scholar]
- Hunkeler, D.; Lichtenvort, K.; Rebitzer, G. Environmental Life Cycle Costing; CRC Press: Boca Raton, FL, USA, 2008. [Google Scholar]
- Bubeck, D. Life Cycle Costing (LCC) im Automobilbau; Verlag Dr. Kovac: Hamburg, Germany, 2002. [Google Scholar]
- Halog, A.; Chan, A. Toward sustainable production in the Canadian oil sands industry. Proceedings of the 13th CIRP International Conference on Life Cycle Engineering, European Union, 31 May–2 June 2006; pp. 131–136.
- Spangenberg, J.H. Environmental space and the prism of sustainability: Frameworks for indicators measuring sustainable development. Ecol. Indic. 2002, 2, 295–309. [Google Scholar]
- Hauschild, M.; Dreyer, L.; Jorgensen, A. Assessing social impacts in a life cycle perspective—Lessons learned. CIRP Ann. Manu. Technol. 2008, 57, 21–24. [Google Scholar]
- Labuschagne, C.; Brent, A. Social indicators for sustainable project and technology life cycle management in the process industry. Int. J. Life Cycle Assess. 2006, 11, 3–15. [Google Scholar]
- Global Reporting Initiative (GRI): Sustainability Reporting Guidelines; GRI: Boston, MA, USA, 2002.
- Jørgensen, A.; Le-Boqc, A.; Nazakina, L.; Hauschild, M. Methodologies for social life cycle assessment. Int. J. Life Cycle Assess. 2008, 13, 96–103. [Google Scholar]
- Jørgensen, A.; Hauschild, M.; Jørgensen, H.; Wangel, A. Relevance and feasibility of social life cycle assessment from a company perspective. Int. J. Life Cycle Assess. 2009, 14, 204–214. [Google Scholar]
- Jørgensen, A.; Finkbeiner, M.; Jørgensen, M.; Hauschild, M. Defining the baseline in social life cycle assessment. Int. J. Life Cycle Assess. 2010, 15, 376–384. [Google Scholar]
- UNEP. Guidelines for Social Life Cycle Assessment of Products; UNEP-SETAC Life-Cycle Initiative: Paris, France, 2009. [Google Scholar]
- Geibler, J.V.; Walbaum, H; Liedke, C. Development of Sustainable Bioprocesses: Modelling and Assessment; John Wiley & Sons, Ltd: Chichester, UK, 2006; pp. 82–113. [Google Scholar]
- Koch, R. The 80/20 Principle: The Secret of Achieving More with Less; Nicholas Brealey Publishing: London, UK, 2001. Available online: http://www.scribd.com/doc/3664882/The-8020-Principle-The-Secret-to-Success-by-Achieving-More-with-Less (accessed on 16 February 2011).
- Kloepffer, W. Life cycle sustainability assessment of products. Int. J. LCA 2008, 13, 89–95. [Google Scholar]
- Kloepffer, W. Life-cycle based sustainability assessment as part of LCM. Proceedings of the 3rd International Conference on Life Cycle Management, Zurich, Switzerland, 27–29 August 2007.
- Ford, A. Modelling the Environment: An Introduction to System Dynamics Modelling of Environmental Systems; Island Press: Washington, DC, USA, 1999; p. 3. [Google Scholar]
- Saeed, K. Slicing a complex problem for systems dynamics modelling. Syst. Dynam. Rev. 1992, 8, 251–262. [Google Scholar]
- Pór, A.; Stahl, J.; Temesi, J. Decision support system for production control: Multiple criteria decision making in practice. Eng. Cost. Prod. Econ. 1990, 20, 213–218. [Google Scholar]
- Spengler, T.; Geldermann, J.; Hähre, S.; Sieverdingbeck, A.; Rentz, O. Development of a multiple criteria based decision support system for environmental assessment of recycling measures in the iron and steel making industry. J. Clean. Prod. 1998, 6, 37–52. [Google Scholar]
- Munda, G. A NAIADE based approach for sustainability benchmarking. Int. J. Environ. Technol. Manage. 2006, 6, 65–78. [Google Scholar]
- Munda, G. Social multi-criteria evaluation: methodological foundations and operational consequences. Eur. J. Oper. Res. 2003, 3, 662–677. [Google Scholar]
- Roy, B. The Outranking Approach and the Foundations of ELECTRE Methods; Bana e Costa, C.A., Ed.; Springer-Verlag: Berlin, Germany, 1991; pp. 155–183. [Google Scholar]
- Brans, J.; Vincke, P. PROMETHEE method for multiple criteria decision-making. Manag. Sci. 1985, 31, 647–656. [Google Scholar]
- Wang, J-J.; Jing, Y-Y.; Zhang, C-F.; Zhao, J-H. Review on multi-criteria decision analysis aid in sustainable energy decision-making. Renew. Sustain. Energ. Rev. 2009, 13, 2263–2278. [Google Scholar]
- Pohekar, S.; Ramachandran, M. Application of multi-criteria decision making to sustainable energy planning—a review. Renew. Sustain. Energ. Rev. 2004, 8, 365–381. [Google Scholar]
- Halog, A.; Wei, B.; Sagisaka, M.; Inaba, A. A Multi-attribute assessment of environmentally-sound electric vehicle battery technologies. J. Ind. Eng. 2004, 1, 40–59. [Google Scholar]
- Halog, A. Selection of Sustainable Product Improvement Alternatives. Ph.D. Dissertation; University of Karlsruhe: Karlsruhe, Germany, 2002. Available online: http://digbib.ubka.uni-karlsruhe.de/volltexte/242002 (accessed on 16 February 2011). [Google Scholar]
- Pineda-Henson, R.; Culaba, A.; Mendoza, G. Evaluating environmental performance of pulp and paper manufacturing using the analytic hierarchy process and life-cycle assessment. J. Ind. Ecolo. 2002, 6, 15–28. [Google Scholar]
- Saaty, T.L. The Analytic Hierarchy Process; McGraw Hill Company: New York, NY, USA, 1980. [Google Scholar]
- Saaty, T.L. How to Structure and Make Choices in Complex Problems. Hum. Syst. Manag. 1982, 3-4, 255–260. [Google Scholar]
- Saaty, T. Theory and Applications of the Analytic Network Process; RWS Publications: Pittsburgh, PA, USA, 2005. [Google Scholar]
- Clift, R. Metrics for supply chain sustainability. Clean Technol. Environ. Polic. 2003, 5, 240–247. [Google Scholar]
- Clift, R.; Wright, L. Relationships between environmental impacts and added value along the supply chain. Technol. Forecast. Soc. Change 2000, 65, 281–295. [Google Scholar]
- Taylor, A.P.; Postlethwaite, D. Overall business impact assessment (OBIA). Proceedings 4th LCA Case Studies Symposium, Brussels, Belgium, 3 December 1996; pp. 181–187.
- Biswas, G.; Clift, R.; Davis, G.; Ehrenfeld, J.; Foerster, R.; Jolliet, O.; Knoepfel, I.; Luterbacher, U.; Russell, D.; Hunkeler, D. Ecometrics: Identification, categorization, and life cycle validation. Int. J. LCA 1998, 3, 184–190. [Google Scholar]
- Emrouznejad, A.; Parker, B.; Tavares, G. Evaluation of research in efficiency and productivity: A survey and analysis of the first 30 years of scholarly literature in DEA. J. Soc.-Econ. Plan. Sci. 2008, 42, 151–157. [Google Scholar]
- Charnes, A.; Cooper, W.; Rhodes, E. Measuring the efficiency of decision making units. Eur. J. Oper. Res. 1978, 43, 429–444. [Google Scholar]
- Lehni, M. Measuring Eco-Efficiency with Cross Comparable Indicators; WBCSD: Geneva, Switzerland, 1999. [Google Scholar]
- Liang, L.; Yang, F.; Cook, W.; Zhu, J. DEA models for supply chain efficiency evaluation. Ann. Oper. Res. 2006, 145, 35–49. [Google Scholar]
- Tyteca, D. Linear programming models for the measurement of environmental performance of firms—concepts and empirical results. J. Prod. Anal. 1997, 8, 183–197. [Google Scholar]
- Tyteca, D. On the measurement of the environmental performance of firms—a literature review and a productive efficiency perspective. J. Environ. Manage. 1996, 46, 281–308. [Google Scholar]
- Kuosmanen, T.; Kortelainen, M. Measuring eco-efficiency of production with data envelopment analysis. J. Ind. Ecol. 2005, 9, 59–72. [Google Scholar]
- Kuosmanen, T.; Kortelainen, M. Data envelopment analysis in environmental valuation: Environmental performance, eco-efficiency and cost-benefit analysis. In discussion papers in Economics; University of Joensuu: Joensuu, Finland No. 21, EconWPA working paper at WUSTL, No. 0409004. , 2004. [Google Scholar]
- Sevkli, M.; Koh, S.; Zaim, S.; Demirbag, M.; Tatoglu, E. An application of data envelopment analytic hierarchy process for supplier selection: A case study of BEKO in Turkey. Int. J. Prod. Res. 2007, 45, 1973–2003. [Google Scholar]
- Belegri-Rompoli, A.; Tsolas, I. Environmental performance measurement at sectoral level: electricity, gas and water. Proceedings of the 8th International Conference on Environmental Science and Technology, Lemmos Island, Greece, 8 September 2003; pp. 77–83.
- Halog, A.; Sagisaka, M.; Inaba, A. Developing Industry-based eco-efficiency measure for Japanese firms using LCA and DEA approaches. Proceedings of 1st International Conference on Eco-efficiency for Sustainability: Quantified Methods for Decision Making, Leiden, The Netherlands, 1–3 April, 2004; p. 59.
- Figge, F.; Hahan, T. Sustainable value added—measuring corporate contributions to sustainability beyond eco-efficiency. Ecol. Econ. 2004, 48, 173–187. [Google Scholar]
- Callens, I.; Tyteca, D. Towards indicators of sustainable development for firms: A productive efficiency perspective. Ecol. Econ. 1999, 28, 41–53. [Google Scholar]
- Holland, J.H.; Miller, J.H. Artificial adaptive agents in economic theory. Amer. Econ. Rev. 1991, 81, 365–371. [Google Scholar]
- Akkermans, H. Emergent supply networks: System dynamics simulation of adaptive supply agents. Proceedings of the 34th Hawaii International Conference on System Sciences, Hawaii, HI, USA, 3–6 January 2001; p. 11.
- Axelrod, R. The Complexity of Cooperation: Agent-Based Models of Competition and Collaboration; Princeton University Press: Princeton, NJ, USA, 1997. [Google Scholar]
- Parker, D.; Manson, S.; Janssen, M.; Hoffmann, M.; Deadman, P. Multi-agent systems for the simulation of land-use and land-cover change: A review. Ann. Assn. Amer. Geogr. 2003, 93, 314–337. [Google Scholar]
- Zhou, Z.; Chan, W.K.; Chow, J.H. Agent-based simulation of electricity market: A survey of tools. Artif. Intell. Rev. 2007, 28, 305–342. [Google Scholar]
- Kempner, R.; Beck, J.; Petrie, J. Design and analysis of bioenergy networks: a complex adaptive system approach. J. Ind. Ecol. 2009, 13, 284–305. [Google Scholar]
- Fox, M.S.; Barbuceanu, M.; Teigen, R. Agent-oriented supply-chain management. Int. J. Flexible Manuf. Syst. 2000, 12, 165–188. [Google Scholar]
- Julka, N.; Karimi, I.; Srinivasan, R. Agent-based supply chain management: A refinery application. Comput. Chem. Eng. 2002, 26, 1771–1781. [Google Scholar]
- Swaminathan, J.M.; Smith, S.F.; Sadeh, N.M. Modelling supply chain dynamics: a multiagent approach. Decision Sci. 1998, 29, 607–632. [Google Scholar]
- Athanasiadis, I.; Mitkas, P.A. A methodology for developing environmental information systems with software agents. Whitestein Ser. Softw. Agent Technol. Auton. Comput. 2009, 119–137. [Google Scholar] [CrossRef]
- Castle, C.J.E.; Crooks, A.T. Principles and concepts of agent-based modelling for developing geospatial simulations; Working Paper 110. Centre For Advanced Spatial Analysis, University College London: London, UK, 2006. [Google Scholar]
- Minar, N.; Burkhart, R.; Langton, C.; Askenazi, M. The Swarm simulation system: A toolkit for building multi-agent simulations; Working Paper 96-06-042. Santa Fe Institute: Santa Fe, NM, USA, 1996. [Google Scholar]
- North, M.J.; Collier, N.T.; Vos, J.R. Experiences creating three implementations of the repast agent modelling toolkit. ACM Trans. Model Comput. Simul. 2006, 16, 1–25. [Google Scholar]
- North, M.; Machal, C. Managing Business Complexity: Discovering Strategic Solutions with Agent-Based Modelling and Simulation; Oxford University Press: Oxford, UK, 2007; p. 221. [Google Scholar]
- Luke, S.; Cioffi-Revilla, C.; Panait, L.; Sullivan, K.; Balan, G. MASON: A multi-agent simulation environment. Simulation 2005, 81, 517–527. [Google Scholar]
- Sterman, J.D. System dynamics modelling: Tools for learning in a complex world. Calif. Manage. Rev. 2001, 43, 8–25. [Google Scholar]
- Borshchev, A.; Filippov, A. From system dynamics and discrete event to practical agent based modelling: Reasons, techniques, tools. Proceedings of The 22nd International Conference of the System Dynamics Society, Oxford, UK, 25–29 July 2004.
- Radzicki, M.; Taylor, R. Origin of System Dynamics: Jay, W. Forrester and the History of System Dynamics. U.S. Department of Energy's Introduction to System Dynamics. Available online: http://www.systemdynamics.org/DL-IntroSysDyn/start.htm (accessed on 22 February 2011).
- Forrester, J.W. Industrial Dynamics; John Wiley & Sons, Inc.: New York, NY, USA, 1961. [Google Scholar]
- Forrester, J.W. Counterintuitive behaviour of social systems. Technol. Rev. 1971, 73, 52–68. [Google Scholar]
- Chan, A.; Hoffman, R.; McInnis, B. The role of systems modelling for sustainable development policy analysis: The case of bio-ethanol. Ecol. Soc. 2004, 9. Article 6. [Google Scholar]
- Riley, C.; Wooley, R.; Sandor, D. Implementing systems engineering in the US. Department of Energy Office of the Biomass Program. Proceedings of International Conference on System of Systems Engineering: SOSE in Service of Energy and Security, San Antonio, TX, USA, 16–18 April 2007; Available online: http://www.nrel.gov/docs/fy07osti/41406.pdf (accessed on 16 February 2011).
- Bush, B.; Duffy, M.; Sandor, D.; Peterson, S. Using system dynamics to model the transition to biofuels in the United States. Proceedings of the Third International Conference on Systems of Systems Engineering Monterey, California, CA, USA, 2–4 June 2008; NREL/CP-150-43153. Available online: http://www.nrel.gov/docs/fy08osti/43153.pdf (accessed on 16th February 2011).
- Sheehan, J.J. Biofuels and the conundrum of sustainability. Curr. Opin. Biotechnol. 2009, 20, 318–324. [Google Scholar]
- Scheffran, J.; BenDor, T. Bioenergy and land use: A spatial-agent dynamic model of energy crop production in Illinois. Int. J. Environ. Pollut. 2009, 39, 4–27. [Google Scholar]
- Franco, C.; Ochoa, M.C.; Florez, A.M. A system dynamics approach to biofuels in Colombia. Proceedings of the 27th International Conference of the System Dynamics Society, Albuquerque, NM, USA, 26–30 July 2009; Available online: http://www.systemdynamics.org/conferences/2009/proceed/papers/P1189.pdf (accessed on 16 February 2011).
- Kim, J.; Xu, M.; Kahhat, R.; Allenby, B.; Williams, E. Designing and assessing a sustainable networked delivery (SND) system: Hybrid business-to-consumer book delivery case study. Environ. Sci. Technol. 2009, 43, 181–187. [Google Scholar]
- Xu, M.; Allenby, B.; Kim, J.; Kahhat, R. A dynamic agent-based analysis for the environmental impacts of conventional and novel book retailing. Environ. Sci. Technol. 2009, 43, 2851–2857. [Google Scholar]
- Kim, J. Sustainability network theory and analysis: Focused on economic, energy and environmental flow network. PhD Dissertation, Arizona State University, Mesa, AZ, USA, December 2008. [Google Scholar]
- Holmberg, J.; Robert, J.H. Backcasting from non-overlapping sustainability principles—A framework for strategic planning. Int. J. Sustain. Dev. World Ecolo. 2000, 7, 1–18. [Google Scholar]
- Holmberg, J.; Lundqvist, U.; Robert, K.; Wackernagel, M. The ecological footprint from a systems perspective of sustainability. Int. J. Sustain. Dev. World Ecolo. 1999, 6, 17–33. [Google Scholar]
- Kuisma, J. Backcasting for Sustainable Strategies in the Energy Sector; IIIEE Report. Lund: Sweden, September 2000; Volume 18. [Google Scholar]
- Robinson, J. Future subjunctive: Backcasting as social learning. Futures 2003, 35, 839–856. [Google Scholar]
- Geurs, K.; Wee, V.B. Backcasting as a tool to develop a sustainable transport scenario assuming emission reductions of 80–90%. Innovation 2000, 13, 47–62. [Google Scholar]
- Dreborg, K.H. Essence of backcasting. Futures 1996, 28, 813–828. [Google Scholar]
- Borbely, M.; Muirhead, J.R.; Graniero, P.A. Backcasting and forecasting biological invasions of inland lakes. Ecol. Appl. 2004, 14, 773–783. [Google Scholar]
- Perlack, R.D.; Wright, L.L.; Turhollow, A.F.; Graham, R.L.; Stokes, B.J.; Erbach, D.C. Biomass as feedstock for a bioenergy and bioproducts industry: The technical feasibility of a billion-ton annual supply; Office of Scientific and Technical Information, Oak Ridge National Laboratory, US Department of Energy: Oak Ridge, TN, USA, 2005 DOE/GO-102005-2135.. [Google Scholar]
- Dickerson, K.; Rubin, J.; Kavkewitz, J. Biomass and biofuels in Maine: Estimating supplies for expanding the forest products industry; Margaret Chase Smith Policy Centre, University of Maine: Orono, ME, USA, 2009; p. 19. [Google Scholar]
- Buchholz, T.; Rametsteiner, E.; Volk, T.; Luzadis, V. Multi criteria analysis for bioenergy systems assessments. Energ. Policy 2009, 37, 484–495. [Google Scholar]
- Buchholz, T.; Luzadis, V.; Volk, T. Sustainability criteria for bioenergy systems: Results form an expert survey. J. Clean. Prod. 2009, 17, S86–S97. [Google Scholar]
- Dhungel, S.; Halog, A.; Neupane, B. A complex systems approach to biofuel supply chain: understanding social, economic and environmental relationships. Proceedings of Life Cycle Assessment X Conference: Bridging Science, Policy and Public, Portland, OR, USA, 4–10 November 2010; Available online: http://www.lcacenter.org/LCAX/abstracts/abstract.php?id=108 (accessed on 16 February 2011).
- MEA. Ecosystems and Human Well-being: Synthesis; Millennium Ecosystem Assessment: Washington, DC, USA, 2005. [Google Scholar]
- Baumgartner, R. Dealing with uncertainty—integrated sustainability assessment based on Fuzzy Logic Sustainable Development and Planning II; WIT Press: Billerica, MA, USA, 2006; Volume 1, p. 2. [Google Scholar]
- Law, A.; Kelton, W. Simulation Modelling and Analysis, 3rd ed.; McGraw-Hill: New York, NY, USA, 2002. [Google Scholar]
- Halog, A.; Sagisaka, M.; Inaba, A. Modelling uncertainties in assessing waste gasification technology. MACRO Rev. Spec. Issue 2003, 16, 251–255. [Google Scholar]
- Huijbregts, M. Dealing with parameter uncertainty and uncertainty due to choices in life cycle assessment. Int. J. Life Cycle Assess. 1998, 3, 343–351. [Google Scholar]
- Miller, J. Active nonlinear tests (ANTs) of complex simulation models. Manage. Sci. 1998, 44, 620–630. [Google Scholar]
- Scholl, H. Agent-based and system dynamics modelling: A call for cross study and joint research. Proceedings of 34th Annual Hawaii International Conference on System Sciences, Maui, HA, USA, 3–6 January 2011; Available online: http://projects.ischool.washington.edu/jscholl/Papers/DTABS01.PDF (accessed on 16 February 2011).
© 2011 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).