- freely available
Sustainability 2013, 5(7), 3035-3059; doi:10.3390/su5073035
Published: 12 July 2013
Abstract: This paper argues that the need for a core “fourth pillar” of sustainability/sustainable development, as demanded in multiple arenas, can no longer be ignored on the grounds of intangibility. Different approaches to this vital but missing pillar (cultural-aesthetic, religious-spiritual, and political-institutional) find common ground in the area of ethical values. While values and aspects based on them are widely assumed to be intangible and immeasurable, we illustrate that it is possible to operationalize them in terms of measurable indicators when they are intersubjectively conceptualized within clearly defined practical contexts. The processes require contextual localization of items, which can nonetheless fit into a generalizable framework. This allows useful measurements to be made, and removes barriers to studying, tracking, comparing, evaluating and correlating values-related dimensions of sustainability. It is advocated that those involved in operationalizing sustainability (especially in the context of creating post-2015 Sustainable Development Goals), should explore the potential for developing indicators to capture some of its less tangible aspects, especially those concerned with ethical values.
1.1. The “Missing Pillar” of Sustainability: A Convergence of Perspectives
There is a significant growing concern in several arenas that the “three-pillar” model of sustainability, consisting of environmental, economic and social dimensions, may be overlooking something of fundamental importance. As highlighted by Littig and Griessler  and more recently by Dahl , there have been several attempts to define this missing dimension as a fourth pillar of sustainability, but it has been variously described as a cultural-aesthetic, political-institutional, or religious-spiritual dimension:
Cultural-aesthetic. A well-established framing of the fourth pillar, or missing dimension, of sustainability conceptualizes it in terms of culture, the arts and/or aesthetics. Jon Hawkes makes this case explicitly in his book The Fourth Pillar of Sustainability: Culture’s Essential Role in Public Planning , where he argues that “cultural vitality”, understood in the sense of “wellbeing, creativity, diversity and innovation”  (p. 25), should be treated as one of the basic requirements of a healthy society. While advocating for community involvement in arts practice, Hawkes makes it clear that he is referring to a broader definition of culture that is not limited to arts and heritage, but encompasses the “whole complex of distinctive spiritual, material, intellectual and emotional features that characterize a society or social group”, as outlined in the 1982 Mexico City Declaration on Cultural Policies .
UNESCO has similarly been active in promoting the cultural perspective, and many of its publications since the 1990s have highlighted the central role of culture in sustainability—either as a “self-standing pillar of sustainable development”  (p. 7) or as a foundation underlying the other three pillars . This has been particularly significant within the context of Education for Sustainable Development (ESD), where the cultural pillar has a strong focus on acknowledging and respecting diverse worldviews, identities and local languages and promoting open dialogue and debate. As discussed by Sacha Kagan in her book Art and Sustainability , there have also been a number of international declarations and processes aimed simultaneously at raising awareness of sustainability within the arts and culture sectors, and at incorporating a cultural-aesthetic dimension into ongoing sustainability discourses. These include, among others, the 2001 “Tutzinger Manifest” (a call issued at a conference on Aesthetics of Sustainability in Tutzing, Germany, for a cultural dimension to be integrated into the Agenda 21 processes arising from the 1992 Rio Earth Summit); the “Agenda 21 for Culture” initiative led by the United Cities and Local Governments initiative, a coalition of local governments from different countries emerging from the 2004 Forum of Local Authorities for Social Inclusion; and the “Culture Futures” conference co-organized in 2009 by prominent civil society organizations in the arts and culture sector, as a parallel to the COP15 UN Climate Conference in Copenhagen. The latter conference has generated ongoing activities and collaborations among the organizations concerned, such as the “Connect2Culture” initiative of the Asia-Europe Foundation.
A “cultural-aesthetic” perspective can also be found among Indigenous communities and their advocates, including the UN Food and Agriculture Organization (FAO), who frame the missing-pillar debate in terms of “cultural integrity”. This term is used to encompass shared values, beliefs and knowledge, as well as more tangible manifestations of culture such as ceremonies and objects [8,9]. Within this context, the United Nations Permanent Forum on Indigenous Issues  (p. 7) has acknowledged the need for culturally appropriate indicators of well-being and sustainability that reflect “true indigenous perspectives such as portraying approaches grounded in wholism [sic] and unique values”. As recently highlighted by Barkin and Lemus , these Indigenous understandings of well-being and sustainability may be rooted in epistemologies that are very different from those of mainstream sustainability discourses.
Political-institutional. The concept of a “political-institutional” fourth pillar is also widely known. Institutional aspects of sustainability were explicitly addressed in the indicator system developed by the Commission on Sustainable Development (CSD) in 1995 to assess implementation of Agenda 21 [12,13,14] as well as being the subject of a dedicated chapter in the Brundtland report, Our Common Future [1,15]. As Spangenberg  explains, institutions are “the result of interpersonal processes, such as communication and co-operation, resulting in information and systems of rules governing the interaction of members of a society”  (p. 104). The development of institutional sustainability indicators is rooted in an understanding of institutions which includes, but is not limited to, organizations: it also encompasses two other categories, namely institutional orientations (norms) and institutional mechanisms (formal systems of rules and procedures, whether administrative, social, political or legal) [13,14].
Beyond the initial Agenda 21 context, the use of the institutional dimension as a fourth pillar of sustainability has gained widespread acceptance within the European Commission and the United Nations. The System of Environmental and Economic Accounting (SEEA) refers directly to “the three-pillars approach (with sometimes a fourth—institutional—pillar)”  (p. 5). The United Nations Division for Sustainable Development also incorporates institutional indicators into its framework of sustainable development indicators [17,18,19].
Religious-spiritual. A third, and much lesser-known, perspective on the missing pillar/dimension of sustainability is rooted in the concept of an awakening global ethical and spiritual consciousness that underpins sustainability transitions [20,21,22]. In his keynote address at the 2010 Earth Charter conference “An Ethical Framework for a Sustainable World” Steven Rockefeller described this emerging consciousness as “in truth the first pillar of a sustainable way of life”, on the grounds that ethical vision and moral courage are essential to generating the political will required for transitions to sustainability  (p.174, emphasis added). A similar sentiment is expressed through a slightly different metaphor in the Interreligious Statement to Rio + 20, developed by religious and spiritual leaders from diverse traditions, which describes ethical/spiritual consciousness as “the foundation of the other three pillars” .
While these three conceptualizations of the missing pillar of sustainability may appear disparate at first sight, we propose that one thing which they all have in common is a concern with human values and how they are manifested in people’s personal and professional lives [8,12,20]. This is not, of course, intended to imply that the dimension of values covers the entire scope of the above perspectives on the fourth pillar of sustainability, as they all encompass multiple constructs which are interrelated in complex ways. Nonetheless, we suggest that values constitute an important and hitherto unrecognized area of common ground between the perspectives, and that there is a strong case for highlighting values as a key element of the less tangible dimension that tends to be omitted from international sustainability discourses.
The term “values” conceals a multiplicity of contested and often conflicting meanings, deriving from many different disciplines of academic research (ranging from moral philosophy to empirical social psychology), and discussions of values span the epistemological divide between the natural and social sciences and the humanities. For the purposes of this paper, however, we have found the twofold definition provided by the Oxford English Dictionary to be helpful for resolving some of the confusion that often surrounds its everyday usage. The dictionary defines values both as “principles or standards of behaviour [sic]” (Definition A) and as “one’s judgement [sic] of what is important in life” (Definition B) . We would argue that Definition A encompasses Definition B, in that the creation of principles or standards for the ways in which people “ought to behave” is inherently rooted in judgments about what is important in life, whether or not these judgments are explicitly articulated. It is the acceptance of a certain world-view, or set of life priorities, that generates the sense of “oughtness”—the compulsion to choose an apparently greater good over an apparently lesser good, or an apparently lesser evil over an apparently greater evil . We note that neither definition requires the use of specific “value-labels” (such as trust, justice, collaboration, etc.): many aspects of our lives which are not directly linked to specific, conceptualized values but can still be values-based, e.g., the education we choose for our children.
Values in the sense of Definition B (people’s judgments about what is important in life) have long been recognized as one of the major transformative forces influencing the ability to fulfill human needs . It has been argued, in this sense, that even natural sciences can never be value-neutral  and that all sustainability indicators are therefore values-based to some extent, even though some (especially in the environmental domain) attempt to hide their underlying values beneath a “façade of objectivity” . We suggest, however, that what is absent from many discussions of sustainability is an explicit consideration of values in the sense of Definition A (principles or standards of behavior), which we will henceforth refer to as “ethical values” for the sake of clarity. Such ethical values can be viewed as linking discussions of cultural integrity and vitality (cultural/aesthetic perspective), personal growth (religious/spiritual perspective) and good governance (political/institutional perspective).
We acknowledge that the use of the “fourth pillar” metaphor to signify the existence of a less tangible dimension to sustainability, with ethical values as one of its key elements, is an over-simplification—especially as ethical values permeate every field of human endeavor represented by the three existing pillars, and the social dimension in particular is intimately concerned with ethical values such as equity and justice . We can envisage alternative metaphors, e.g., ethical values as a foundation or lintel for the three pillars, as a weft running through them, or as a spiral that winds around them—in each case touching each of them and linking them together, but also incorporating something extra. Our main reason for promoting the fourth pillar metaphor, instead of any other viable metaphor, is to advocate for the inclusion of ethical values in international sustainability discourses on equal terms with the three existing pillars. This, we suggest, could facilitate the mainstreaming of the concept of ethical values and reduce the likelihood of its being dismissed as a trivial detail.
1.2. Ethical Values in International Sustainability Discourses
The theme of ethical values as a crucial dimension of sustainability is taken up strongly in the Earth Charter [20,21,29] and the United Nations Millennium Declaration , which both call explicitly for an “ethical framework” for sustainability and list specific values which they associate with it. The text of these documents, quoted in Table 1 below, makes it clear that they are referring to values in the sense of ethical principles or standards for behavior—albeit underpinned by shared understandings of what is important in life. The Millennium Declaration also makes the first attempt to provide definitions of some of these ESD-related ethical values, albeit in a very general way. A similar perspective is evident in the United Nations Educational, Scientific and Cultural Organization (UNESCO) International Implementation Scheme for the 2005–2014 Decade of Education for Sustainable Development (DESD), where both of the above meanings are clearly evident . This document states, for example, that “the basic vision for the DESD is a world where everyone has the opportunity to benefit from education and learn the values, behaviour (sic) and lifestyles required for a sustainable future and for positive societal transformation” . De Leo  has conducted a content analysis of 22 international documents relevant to ESD, dating from 1945 to 2006 inclusive, and identified from them 16 frequently mentioned “global values” (along with 17 less frequent ones). All of the frequently mentioned values, except “freedom”, are also mentioned in the International Implementation Scheme for the DESD.
By contrast, in the 2002 Rio + 10 report  the word “values” is mentioned only once, in a small section reporting on a roundtable that refers specifically to the values of democracy as freedom, equality, tolerance, and respect for nature. In the final outcome document from Rio + 20, The Future We Want , the concept of values (either in the sense of ethical principles or of value judgments) disappears altogether. The Rio + 20 document does not explicitly mention the word “values” at all, except in a third sense relating to worth (ecological, economic and other “values of biological diversity”).
It could be argued that the momentum generated around ethical values in the international sustainability discourse at the turn of the millennium has already been lost. An alternative interpretation is that the Commission for Sustainable Development (CSD) process and the Millennium Declaration/Earth Charter process actually represent parallel discourses. Koroneos and Rokos suggest that the paradigm of “sustainable development” espoused by the CSD process has already been co-opted by the neoliberal economic growth agenda, and outline an alternative paradigm of development that is fundamentally rooted in ethics and human well-being, which they term “(worth-living) integrated development” . This latter paradigm may be more compatible with the understanding espoused in the Millennium Declaration and Earth Charter, as well as the ethical values-based position on ESD that has been adopted by UNESCO.
Whatever the underlying reasons, the lack of any reference to ethical values or principles in the Rio + 20 outcome document is a cause for great concern. Since the Rio + 20 outcome document is clearly intended to guide the creation of the forthcoming Sustainable Development Goals (SDG), there is a strong likelihood that reference to ethical principles of sustainable behavior will be entirely omitted from the SDG process, unless early and decisive action is taken by the relevant institutions to identify appropriate ethical values-related goals, targets and indicators.
|Table 1. “Values” in international sustainability/sustainable development documents, 2000–2012.|
|Source||Values of Sustainability/Sustainable Development (SD)||How described|
|Earth Charter||Respect and Care for the Community of Life |
Social and Economic Justice
Democracy, Non-Violence and Peace
|“We urgently need a shared vision of basic values to provide an ethical foundation for the emerging world community”  (p. 1)|
“…we affirm the following interdependent principles for a sustainable way of life as a common standard by which the conduct of all individuals, organizations, businesses, governments, and transnational institutions is to be guided and assessed”  (p. 1)
|UN Millennium Declaration (2000)||Freedom, Equality|
Respect for Nature
|“We consider certain fundamental values to be essential to international relations in the twenty-first century.”  (Section 6)|
|“Global values” (1945–2006)||Equality (561)|
|This list contains the 16 most frequent “global values” identified through a content analysis of 22 international documents issued between 1945 and 2006 inclusive . Each word in the list represents a cluster of similar concepts or meanings (e.g., “Peace” represents “peace, non-violence, harmony, social cohesion”). The numbers in brackets refer to the number of times that the respective value cluster was mentioned in the 22 documents. Less frequently mentioned values were love, nurturing, privacy, confidentiality, informed/free consent, innovation, creativity, imagination, empowerment, generosity, gratitude, humility, wisdom, resilience, hope, stability, and reverence (i.e., for life and the environment). These values do not appear as lists in the original documents; their collation into list form has been done by De Leo.|
|Final Report from Rio + 10 (2002)||Freedom|
Respect for Nature
|“Some participants [in Round Table 3] highlighted links among environment, trade, peace accords, military arms reduction, the implementation of the Monterrey agreement and developing the values of democracy within a sustainable development framework. They supported adherence to the principles…”  (p. 127)|
|Final Report from Rio + 20 (2012)||None listed||“We reaffirm the intrinsic value of biological diversity, as well as the ecological, genetic, social, economic, scientific, educational, cultural, recreational and aesthetic values of biological diversity…”  (p. 38)|
In this paper, we aim to contribute to ongoing conversations around the nature and roles of post-2015 Sustainable Development Goals by examining a logical argument for the exclusion of ethical values from sustainability assessment processes (the “immeasurability” argument); illustrating that it is based on a false premise; and replacing it with an alternative logical argument (the “context-specific measurability” argument). We also discuss practical considerations relating to the introduction of values-based indicators, and provide an illustrative example of their use.
1.3. Toward Ethical Values-Based Indicators
Existing frameworks of sustainability indicators specifically acknowledge the importance of “values” in the sense of shared priorities (our Definition B). The European Reference Framework for Sustainable Cities (RFSC), for example, refers directly to the importance of building an integrated vision for sustainable development based on clear priorities and objectives, identified through processes of consultation with multiple stakeholders. Indicators that explicitly operationalize ethical values, in the sense of measuring the enactment of widely accepted principles or standards of behavior, are, however, currently lacking.
In the light of the continuing high-level process to develop a global set of post-2015 Sustainable Development Goals, it is timely to ask why, given the widespread recognition of the importance of ethical values for sustainability transitions, the ethical dimension has so often been overlooked in the development of goals and indicators . Even in the arena of institutional sustainability, where the question of “good governance” encompasses many aspects that could be considered values-based, efforts to develop indicators have been fraught with problems. In an assessment of Agenda 21 implementation conducted by the Commission on Sustainable Development, for example, a full 60% of the proposed institutional indicators were dropped before the final version, leaving the institutional dimension with only a third as many indicators as each of the other three dimensions. Many of the indicators dropped from the initial draft had been perceived during field testing to be unclear, irrelevant, lacking data, or in need of further development [10,11]. Those indicators that remained did not adequately address the values aspect of institutions: Spangenberg  criticizes the Agenda 21 assessment process for using an incomplete definition of “institutions”, often treating the term as synonymous with “organizations”, and not going far enough to operationalize less tangible institutions such as values, norms and informal rules.
The omission of some draft indicators of institutional sustainability because of challenges with data availability [14,36] hints at a broader concern about contemporary sustainability assessment. It might be assumed by a lay person that indicator development would precede data collection, but as indicator specialists will attest, the reverse is often true. As noted by McCool and Stankey  (p. 295), in the absence of broad public debate about what constitutes sustainability, efforts to develop new indicators are “guided more by what can be measured (a technical issue) than by what should be measured (a normative issue)”. Indicator development thus appears to be severely constrained, whether by a failure of imagination, resource provision, or both: no matter how important something might be to the public, if it is not currently measured for other purposes (e.g., government statistics) there may be less interest in exploring whether it might be measurable.
Concerns about data availability do not, however, fully explain the lack of investment in values-based sustainability indicators. Another possible reason is given by Dahl  (p. 16) when he explicitly asserts, citing the seminal work of Hitlin and Piliavin , that no values-based indicators have yet been developed because values are “difficult to define and measure, with few widely accepted or standardized methodologies”. In lay discourse and a large body of academic literature, especially in the humanities, values tend to be characterized as highly subjective, subconscious, intangible, affective, context-bound and/or dynamic in nature, with an underlying assumption that there can never be any scientifically valid way of “measuring” them. Redclift and Benton , for example, state that people’s values are “negotiated, transitory and sometimes contradictory”, while the popular author Daniel Goleman powerfully conveys the affective nature of personal values by describing them as “not lofty abstractions, but intimate credos that we may never quite articulate in words so much as feel”  (p. 57). Meglino and Ravlin  (p. 360) similarly refer to values as being “less than totally conscious, somewhat below an individual’s level of complete awareness”. These widely shared subjectivist ontological assumptions about values appear to bring them into direct conflict with concepts of measurement: one cannot envisage measuring something which can neither be unambiguously conceptualized, nor adequately operationalized [42,43].
Does it matter, then, if there are no useful indicators for internationally advocated ethical values such as those listed in Table 1? We would argue that it matters greatly, because indicators often have conceptual and symbolic uses far beyond their instrumental uses [44,45,46]: they do not merely reflect what is important in society, but also contribute to defining what is perceived as important [47,48]. The current absence of ethical values-based indicators may contribute to perpetuating a situation in which “nation-states are “managed” to enhance GDP ahead of almost all other concerns”  (p. 225), while thorny issues of moral accountability are neatly evaded. Conversely, it might be expected that if values-based dimensions of sustainability were to be systematically assessed, one result might be the creation of new political norms (c.f., ) that tend to prioritize values such as equity, tolerance, justice and respect for nature at global, national and local levels.
In the next section, we explore the theoretical grounding for developing useful sustainability indicators based on ethical values, which can help to clarify some elements of the missing pillar of sustainability and render them measurable. We first distinguish between values espousal, which is widely measured through survey instruments in the empirical tradition of social psychology, and values enactment, which has not yet been systematically operationalized. Focusing on values enactment, we then outline a logical argument for excluding it from sustainability assessment discourses, and illustrate that this argument is based on a false premise. In Section 3 we outline an alternative conceptualization focused on the development of intersubjectively valid (rather than objectively valid) values-based indicators for specific practical contexts, and highlight the immediate and significant implications for the SDG process. Finally, in Section 4, we recommend initial steps that could be taken immediately to bring this missing dimension into the SDG agenda.
2. Are Values Measurable?
2.1. What Do We Mean by “Values”? Espousal Versus Enactment
The suggestion that values are too intangible to be measured might be strongly disputed by researchers in the empirical tradition of social psychology, where values survey instruments (addressing values in the explicit sense of “what matters to people”, but also implicitly ethical principles and standards) have been developed and widely used for many decades. Rokeach, for example, in his widely cited “Rokeach Value Survey” (RVS), claims to represent the entire domain of human values in two separate lists of 18 words or short phrases representing, respectively, modes of behavior and idealized conditions (“end-states”) of existence. Respondents rank each set of items in order of importance as principles that guide their lives, and the resulting rankings are interpreted as indicators of the individual’s personal values [51,52]. The Schwartz Values Survey (SVS) is broadly similar, but uses 56 predefined value items and replaces the ranking activity with a rating scale [53,54,55,56,57]. There are numerous other survey approaches which, while differing in the detail of their operationalization, share the assumption that an individual’s personal values exist as discrete verifiable constructs which can be deduced from their responses to a questionnaire [58,59,60,61,62,63,64].
To understand why values surveys in the Rokeach/Schwartz tradition might not be sufficient to solve the problem of values-based sustainability indicators, it is important to appreciate that values can manifest themselves both through discourse (what people say) and overt action (what they do). In this paper, we follow Gruys et al.  in using the term “values espousal” to refer to the use of values-related vocabulary in spoken, written and/or multimedia discourses, and “values enactment” to describe situations in which values are “lived out or enacted [by individuals] through their specific actions and behaviours [sic]”—drawing on the earlier work of Argyris and Schön . This formulation of “values enactment” and “values espousal” is derived from research in social psychology, but has parallels with work carried out in other disciplines, such as sociology—notably Bernard Lahire’s  duality of “dispositions to act” and “dispositions to believe”, which in turn draws on the work of Bourdieu .)
As noted by Schlater and Sontag  (p. 5), there is often a mismatch at the individual level between the public espousal of values in discourse and their enactment in behavior: “A person may “talk” the value but not implement it in action, or a person may act in accordance with a value but not subscribe to it verbally.” Values surveys cannot measure enactment of values by individuals, organizations or states, but only what they are willing to articulate verbally: they do not offer any way of identifying whether there is a mismatch between the values implied by respondents” survey responses and those manifested in their real-life actions (Figure 1). We are not suggesting that what people say about their values is unimportant, but rather that attempts to develop useful values-based sustainability indicators should examine the question of enactment (e.g., by observing behavior or conducting surveys of peers) instead of relying entirely on self-report surveys.
2.2. The Theoretical Possibility of Measuring Values Enactment
We turn now to the theoretical question of whether values enactment can ever be measured in a meaningful way through the use of indicators. To do this, we critically examine a logical argument for the immeasurability of values enactment, which might be used implicitly to block investment in the exploration of values-based indicators. We first present the argument in full, then investigate the truth of each of its premises, and finally determine whether the logical reasoning leading from the premises to the conclusion is sound (This analytical approach is adapted from Thwink.org ).
The first premise, as implied by Dahl  in his comment that values are “difficult to define”, is that obtaining an objective definition of values enactment, or of the enactment of a specific value, is impossible see also [38,39,40]. We define “objective”, for this purpose, as existing independently of individual human understandings. The second, a basic assumption underlying indicator development processes, is that an objective definition is required in order to operationalize a concept as measurable indicators . As illustrated in Figure 2, these two premises taken together lead logically to the conclusion that values enactment cannot be operationalized as measurable indicators—and therefore cannot be measured.
2.2.1. Can Values Enactment be Objectively Defined?
Earl Babbie  (p. 128) argues that values such as “compassion” can neither be objectively defined (Premise 1), nor objectively measured. He illustrates this by using the concept of a “mental file sheet” to represent an individual’s conception of how a value might be enacted. If, for example, researcher X observes a subject (“Pat”) performing certain actions that X associates with the word “compassionate” (i.e., actions that appear on X’s own mental file sheet for “compassionate”, such as putting a little bird back into its nest after finding it on the ground, or visiting a children’s hospital at Christmas), she will conclude that Pat is compassionate. If, on the other hand, researcher Y does not observe in Pat’s behavior any of the actions that appear on his own “compassionate” file sheet, and also notices Pat performing other actions that he regards as indicative of a lack of compassion (e.g., refusing to donate money to a campaign to save whales from extinction), he will conclude that Pat is not compassionate. Babbie then comments:
“We can measure, for example, whether Pat actually puts the little bird back in its nest, visits the hospital on Christmas…or refuses to contribute to saving the whales. All of those behaviors exist, so we can measure them. But is Pat really compassionate? We can’t answer that question: we can’t measure compassion in any objective sense, because compassion doesn’t exist in the sense that those things I just described exist.”
Further evidence from cross-cultural values studies supports the truth of Premise 1, that no objective definition of values enactment is possible, because people’s personal understandings of values-related words and phrases (“value-labels”) are heavily influenced by both their cultural contexts and their particular life experiences [72,73,74]. This point is similarly illustrated by literature in management and organization studies, where differences in the ways in which value-labels are understood by managers and employees can contribute to significant problems within organizations—even leading directly, in one case, to the resignation of a senior executive [65,75,76]. We therefore conclude that Premise 1 is true and that values enactment cannot be objectively defined.
2.2.2. Is Objective Definition a Prerequisite for Operationalization and Measurement?
The quest for indicators is, at first glance, inherently objectivist. The very word “indicator” hints at its realist ontological assumptions: namely that there is an underlying “theoretical variable” that exists in objective reality, and that its presence can be unambiguously indicated by one or more, similarly existent, “observable variables” . Premise 2 is entirely consistent with this paradigm of indicator development, but to accept the premise as true is to deny the possibility of any alternative way of understanding indicators.
In the social sciences, as highlighted by Babbie , “most of the variables we want to study don’t exist in the way that rocks exist. Instead, they are made up. Moreover, they seldom have a single, unambiguous meaning.” This does not, however, mean that nothing can ever be operationalized or measured. Rather, useful constructs can be created through mutual agreement for the purpose of communication and research—a process termed “conceptualization”. What this entails is not objectivity but intersubjectivity, a complex and multi-layered concept that we might summarize for the purposes of this argument as the emergence of a human “interworld…of shared meaning that transcends individual consciousness”  (p. 4). This shared meaning emerges in a collaborative context through dialogical interactions grounded in a common lived experience, such as a jointly undertaken practical activity .
To extend Babbie’s  earlier example: if the two researchers in question were employed in a teaching hospital that explicitly aimed to train nurses to be compassionate towards patients, they might already have a degree of shared understanding about what a “compassionate” nurse should do (or avoid doing). Thus, their “mental file sheets”, rather than being completely different, would overlap to a certain extent. Through a formal process of dialogue with hospital stakeholders, they could translate this informal shared understanding into specific measurable indicators and assessment tools, and hence evaluate the extent to which “Pat”—as a final-year student nurse—is enacting the value of compassion in the sense desired by the hospital. It is true that these particular indicators of compassion are unlikely to be generalizable universally, and that they would not encompass every possible aspect of enacting compassion. There is no doubt, however, that they could be locally well-defined; and one can envision that they might provide the hospital stakeholders with useful information about the success or failure of Pat’s training.
We can thus conclude that Premise 2 is false, i.e., an objective definition of a value such as “compassion” or “respect for nature” is not a prerequisite for creating useful indicators of its enactment. Indeed, much of the measurement-based work conducted in the social sciences negates Premise 2 . Rather, what is important is that the value is intersubjectively defined in relation to a specific, bounded practical context  and that the intersubjective definition is accepted by the individuals involved as a useful, if incomplete, working definition.
This implies that “values enactment” is measurable within a specified context, provided that certain parameters are predefined, and leads us to propose the replacement of the “immeasurability” argument with an alternative logical argument (which we term the “context-specific measurability” argument) that opens the way for the development of values-based sustainability indicators. This argument is summarized in Figure 3, below. In the following section, we briefly present an illustrative example from our own work in support of this theoretical argument.
3. Developing and Using Values-Based Indicators: An Illustrative Example
We have asserted that through a process of intersubjective conceptualization within a clearly defined context, the enactment of human values (previously regarded as intangible) can be operationalized and measured. In this section, we will provide a practical example, derived from a collaborative international research project, to illustrate how processes of conceptualization and operationalization of values-related dimensions of sustainability have been used to develop project-level indicators.
3.1. Background: The ESDinds Project
The ESDinds project, which aimed to develop values-based indicators and assessment tools for civil society organizations promoting education for sustainable development, was funded by the European Union Seventh Framework Programme (FP7) from 2009–2011 [79,80,81,82,83]. The project brought together representatives of two academic research institutions and four civil society organizations (CSOs) as equal partners, and was innovative in the extent of decision-making power granted to the CSO partners within the research consortium.
The first phase, which we termed values elicitation, comprised the identification of a pool of value-labels and related pilot indicators (“proto-indicators”) through content analysis of a large qualitative dataset generated through case study research, key informant interviews, workshops and document analysis within six “source” CSOs . As the initial data analysis generated a very long list of values that the CSOs regarded as important for their work, prioritization was carried out on the basis of coding frequency to generate a list of five values with their associated proto-indicators: integrity, trustworthiness, unity in diversity, empowerment and justice. A sixth value, care and respect for the community of life, was also added after two members of the consortium objected that the overall set of values was incomplete without it. A total of 177 draft proto-indicators were intersubjectively chosen by representatives from all source CSOs, from the much larger number extracted from the data set for these six values .
In the second phase, exploratory field work, the peer-elicited proto-indicators were field-tested with “user” CSO partners acting as “critical friends” . Extensive testing of the indicators was conducted in 15 organizations, each of which selected between 3 and 25 indicators to measure locally [81,83]. The feedback was used to improve the relevance/importance, validity, comprehensibility and measurability/usability of the indicators in a variety of different civil society contexts, in accordance with current recommendations for developing sustainability indicators . Through this process, the indicators were discovered to be very broadly relevant across diverse organizational and cultural contexts, and to be a sufficiently large pool to construct indicator sets for several other common human values of importance to CSOs [80,81]. Another important finding was that the usefulness of the indicators was greatly enhanced by localization, i.e., the flexibility to modify the wording to suit the specific practical context: although this also has the disadvantage of precluding direct comparisons between different organizations, it had the advantage of providing excellent face validity. Furthermore, generalizability was not entirely lost as each indicator “item” could still have measures compared qualitatively across organizations or time or circumstances. Once an intersubjective local consensus was reached for those indicators chosen to be of priority locally, devising means of obtaining measures of them became a feasible task. That is, the localization provided the boundary conditions which allowed clear specifications for useful measures to be devised.
One example of the use of this approach is given below in Section 3.2, where we illustrate the use of values-based indicators developed through the ESDinds project to evaluate an online course in sustainability leadership. A separate evaluation of the ESDinds method for purposes of project monitoring and evaluation in three different organizations is described in our earlier work , and there are many other possible applications for these indicators at different levels and in different contexts. The purpose of the case study in this specific paper is to demonstrate the fallacy of the immeasurability argument for enacted values.
3.2. Measuring Values Enactment at the Project Level
One example of the many applications for the indicators developed during the ESDinds project was a values-focused evaluation for a semester-long online course in sustainability leadership. The course seeks to train a new generation of sustainability leaders, with the specific mandate to “promote values in relation to a shared ethics for sustainability” and build capacity in project planning, management, fundraising and social media utilisation . The aim of using the ESDinds toolkit to evaluate the course was to assess the extent to which specific ethical/spiritual values, associated with sustainability leadership and the Earth Charter, were present in (a) the course content; (b) the course implementation; and (c) participants” behaviour during and, where possible, after the course. According to Conclusion #1 in Figure 2, such values-based measurements would not be possible.
The first step in this evaluation was a workshop exercise in which the course facilitators intersubjectively identified and defined which of their values they wanted to assess the presence of, using their own local terminology. In the second step, facilitators read through the full reference list of ESDinds indicators and selected 49 indicators that they felt represented their locally defined value concepts and were highly relevant to the course, without concerning themselves about how they might be assessed. In the third step, the group re-read the indicators again as a set, reduced the list to 40 indicators on the basis that some of them were felt to be duplicated, and reflected on which assessment methods might be appropriate.
A group consultation approach was then used to design a mixed-methods assessment strategy to provide measurements of the 40 indicators. The final chosen (localised) assessment methods were, first, developing a participant questionnaire with both open-ended and multiple-choice questions; and second, conducting a qualitative content analysis of participants” feedback (obtained through the course’s Facebook page and inbuilt feedback mechanism) and of their submitted assignments, which included proposals for sustainability projects in their home communities or elsewhere. These methods were locally deemed sufficient to provide “measures” of local values, for the given context and purpose. The quantitative findings from the survey questionnaire were then converted to qualitative data, and the entire dataset was analysed in terms of participants” and/or facilitators” enactment of each of the five locally defined value clusters: (a) Empowerment; (b) Participation; (c) Innovative Thinking; (d) Equality, Inclusiveness, and Tolerance; and (e) Transformation for Sustainability and Universal Responsibility. The project coordinator felt that the evaluation findings provided the facilitator group with a deep qualitative understanding of how each of the above mentioned values was enacted in the online course. They provided confirmation of its overall effectiveness in catalysing value change towards sustainability, and also highlighted several ways in which the course content or delivery could be improved in order to score more highly on specific values-based indicators.
While the small sample size for the online course meant that it was not necessary or desirable to quantify the data in this particular case, the intersubjective approach to values measurement does allow for quantification and statistical analysis where appropriate. In a larger study, for example, it would be possible to pre-test the questionnaire for internal consistency (e.g., Cronbach’s alpha) and then intersubjectively determine benchmarks. A group might decide, for example, to rate an indicator as “green/good” if more than 75% of responses to an indicator are positive (according to their own collectively agreed definition of what would constitute a positive response), amber/“satisfactory” if 50–75% of responses are positive, and red/‟unsatisfactory” if less than 50% of responses are positive. The ratings for individual indicators could then be aggregated to give an overall rating for the value. We are working on developing quantitative assessment tools for measuring specific pro-sustainability values within a higher education context. The above example illustrates that it is possible to operationalize human values intersubjectively through dialogue, by building consensus around the linkage of locally defined value concepts to specific measurable indicators. These indicators may be drawn directly, and/or modified, from a generic list such as that generated by the ESDinds project. The local meaning of each value-label (e.g., “empowerment” or “equality”) thus becomes defined by consensus-validated referents (behaviours, perceptions and aspirations) within the clearly stated context, in this case the online course. We are not claiming, of course, to have defined “empowerment” in a universally applicable way, but only to have defined the type of empowerment that the facilitators of this specific course were seeking to achieve. However, the overlap between multiple intersubjective definitions of values enactment would be an interesting arena for future research, especially where the contexts are broadly similar.What this means is that named ethical and spiritual values can be “measured” in a useful and locally valid way, through further dialogic processes of devising appropriate data collection and analysis strategies and establishing benchmarks where necessary. According to the requirements of the situation, these strategies may be qualitative, quantitative, or mixed in nature, and may involve a combination of standard methods (e.g., survey questionnaires) and innovative context-specific methods (in this case, content analysis of comments left on the intrinsic feedback mechanism built into the online course platform).
Our empirical findings therefore support the theoretical argument made in Section 2.2.2., namely that it is possible to obtain an intersubjective definition of values enactment within a defined local context (Premise 3) and that the intersubjective definition is sufficient to allow operationalization and measurement of values in that context (Premise 4). Since the measurements obtained in the case study were useful to the team of facilitators, we suggest that Conclusion 2 might be enhanced by the addition of the word “usefully”, thus: “Values enactment can be usefully operationalized as measurable indicators within a clearly defined practical context”.
4.1. Potential for the Further Development of Values-Based Indicators
In this paper, we have illustrated that there is no actual theoretical barrier to the measurement of ethical values, which constitute a key element of the “missing pillar” of sustainability. Although objective and universally accepted definitions of values may be impossible to attain, we introduce work where a useful set of localizable values-based indicators has been successfully developed and utilized at the project level, through a process of intersubjective conceptualization. We propose, therefore, that it is no longer valid to argue that values-based aspects of sustainable development cannot be usefully measured and that, in the light of its inherent flexibility, the ESDinds method could be adapted for developing values-based sustainability indicators in wider contexts. This has significant implications, not only for emerging transnational grassroots movements such as the Earth Charter Initiative  that are keen to understand the strengths and weaknesses of their existing efforts, but also for the entire global apparatus dedicated to defining and measuring sustainability. Even if difficulties were to be found in adapting this exact method, its success thus far may be regarded as “proof of concept” that indicators can be developed for less tangible constructs than might previously have been imagined.
We strongly recommend, therefore, that the institutions tasked with developing Sustainable Development Goals, targets and indicators should take time, at this critical juncture, to pause and reframe the sustainability assessment process. We echo McCool and Stankey  in calling for a shift away from a convenience-driven technical approach (“what can be measured”, using the methods and datasets that are currently available), towards a normative approach based on creative and critical thinking (“what should be measured”). In our view, what is required at this stage is not mere accumulation of indicators , but greater efforts to conceptualize and operationalize “sustainability” from scratch in diverse contexts where explicit discourses of ethical values already exist—including grassroots transnational networks such as the Earth Charter Initiative that promote a clear set of principles, faith-based organizations which articulate values, and Indigenous communities where customary laws incorporating shared values are known and respected.
While this process would almost certainly benefit from some degree of global coordination, it is imperative to avoid mistranslating our call for a “global quest for values-based indicators” as a “quest for global values-based indicators”. Following Hulme , we suggest that a one-size-fits-all approach on a global scale is unlikely to yield useful results, especially because intersubjective processes tend to be inherently rooted in local contexts of shared practical activity . What may be more helpful is a polycentric approach, based on the creation of multiple, diverse, peer-elicited, indicator sets and assessment tools [86,87]. It might be useful, for example, to create reference sets of indicators and survey questionnaires which can be used in their standard forms to generate national-level statistics, but can also be tailored to local contexts by CSOs, local government institutions, religious groups, companies and Indigenous communities as part of an emerging culture of self-evaluation, learning and change. Awareness of the need for such contextualization for practical purposes has been raised previously .
One possible approach would be to begin by developing multiple small-scale frameworks of values-based indicators, beginning at the project and organization levels, and then (a) expanding their reach vertically to neighborhood, city, district, regional and national levels; and/or (b) expanding their reach horizontally via transnational grassroots networks of CSOs, such as the Earth Charter Initiative. In each of these cases, the initial exploration of shared values (in the sense of explicit or tacit principles/standards of behavior applicable within the defined contexts) could be conducted through surveys or interviews that would be statistically representative of the respective populations. Subsequent stages of indicator development could then use approaches such as citizens’ juries to check candidate indicators for perceived relevance, comprehensibility and usefulness. If there is initial resistance to the concept of values-based indicators on the part of decision-makers, one strand of research might initially focus on developing standardized assessment tools (e.g., survey instruments tested for internal consistency) for use at national and global levels, even while CSOs continue to explore flexible approaches tailored to their local realities. Such standardized tools might help to build confidence in the ability of values-based approaches to deliver rigorous evidence to inform decision-making, thereby providing a starting point from which more complex, participatory and mixed-methods approaches can subsequently be developed.
We accept that these suggested processes for developing values-based indicators are potentially complex and may pose implementation challenges, and that they contrast sharply with the inherent reductionism and top-down nature of many conventional processes for creating goals and indicators. The underlying principles of participation, co-design and including multiple stakeholder perspectives are not, however, without precedent. Combinations of “top-down” and “bottom-up” approaches to indicator development have already been employed by Reed, Fraser and colleagues, e.g., in the context of developing rangeland degradation indicators with indigenous herders in the Kalahari [84,88,89,90] and creating well-being assessments with stakeholders in coastal British Columbia . We firmly believe, furthermore, that the centrality of ethical values to sustainability—as attested by the Earth Charter and Millennium Declaration, as well as the work of UNESCO—justifies substantial investment in this area. This view is partially supported by the widespread enthusiasm for values-based indicators that has been generated since the conclusion of the ESDinds project in December 2010. At the time of writing, the web platform created through the project has received over 8000 hits from 138 countries, and has generated an online community of interest with 143 members. Of these, 38% (n = 54) have engaged actively with the platform by marking (and, in many cases, customizing) those indicators that they find relevant to their work. The fact that only 35% (n = 50) of the total membership and 28% (n = 15) of those selecting indicators describe their affiliation as “non-profit, charitable or humanitarian organization”, with the others variously describing themselves as private companies (including social enterprises), faith-based organizations, public sector organizations, academic or educational institutions, families, informal community groups or individuals, attests to the usefulness and relevance of ESDinds values-based indicators beyond the specific sector in which they were developed (authors’ unpublished results).
The political challenges of reframing sustainability to give due consideration to the enactment of ethical values should not be underestimated or trivialized. There are ongoing controversies surrounding the term “sustainable development” in international discourse, including the critique of the implicit economic growth model and its relationship to powerful vested interests within the corporate sector [24,39,86]. Those debates could be seen as existing in a separate plane, but generating parallel arguments that are highly relevant to the plane of sustainability assessment. We maintain, however, that at this critical time in history it is imperative to create a space—however loosely defined—within the Sustainable Development Goals process to allow for subsequent operationalization and measurement of ethical values. This would enable relevant research and practice to be supported, even while the broader and more complex questions of politics, ideologies, power relations and the appropriateness of the term “sustainable development” continue to be debated in multiple arenas. If this opportunity is missed now, it may become increasingly difficult for the voices of fourth-pillar advocates (of whatever persuasion) to be heard in the future.
4.2. Mitigating the Risk of Perverse Effects
Beyond the practical implementation issues that we have already discussed, another concern is the possibility of perverse effects arising from the introduction of values-based indicators. As in the case of other types of indicators, even though the goal of their introduction is to increase rationality in decision-making, there may be some potential for forms of use which undermine this goal. Political use might, for example, lead to instrumentalization and data manipulation; while tactical use might result in the selective communication of results on the basis of personal interest . The flexible, localizable and largely qualitative nature of values-based indicators may render them particularly susceptible to misuse, and even if they are used as intended, care will need to be taken to avoid social desirability response bias when using assessment tools such as survey questionnaires or interview schedules [93,94]. One important feature of the ESDinds system is the use of mixed methods, ideally including at least one method that does not rely on self-report data, in order to reduce the overall effect of social desirability bias when measuring a particular indicator (as in the example above, where questionnaires were supplemented by a qualitative analysis of text that had already been submitted to an online platform). Furthermore, multiple linked indicators can be combined in order to provide information about a specific ethical value or cluster of values.
Within CSOs, a study of eight organizations where the ESDinds indicators have been used (including the example described above) identified a diverse range of positive effects, and no known negative effects (authors” unpublished results). It is possible, of course, that the respondents may have forgotten the perverse effects, chosen not to mention them, or failed to recognize them as attributable to the indicators. It is also possible that perverse effects took longer to become evident than positive effects, and had not yet emerged at the time of the study (3–6 months after the application of the indicators in each case). However, we suggest that in these organizations there may have been additional factors that contributed to the generation of positive rather than negative effects: (i) a shared understanding that the purpose of indicators is to assist local stakeholders to improve the effectiveness of the organization or project, rather than to enable external assessors to criticize its shortcomings; and (ii) a participatory approach in which local stakeholders, in this mode of learning, collaborate in identifying those indicators and assessment tools that they regard as relevant, important and interesting within their local context [80,81]. Further research is necessary to determine whether any aspect of this learning is transferable to higher levels of indicator use, such as a neighborhood, city or local authority.
4.3. Values-Based Indicators, Transdisciplinarity and Sustainability
The ESDinds research project described in this paper draws on a currently dispersed but vast body of work on values, participation and iterative co-learning. This work does not sit comfortably within either the “objectivist” or “subjectivist” paradigms, but rather, aims to promote interdisciplinary learning at the interface between the social and natural sciences  in defiance of the apparent dichotomy between them. It could also be described as transdisciplinary research, i.e., research that looks not only across and between disciplines but also beyond them—at least in the sense in which transdisciplinarity is understood by authors such as Burger  and Gibbons and colleagues , as crossing the boundaries between “science” and “society”. According to this view, adopting a transdisciplinary approach means facilitating the deep participation of non-scientific stakeholders in research and the “direct application of scientific knowledge in both political decision-making and societal problem-solving” [7,96]. The focus of ESDinds was placed on societal problem-solving from the start, with the overarching criterion for decision-making at each point in the process being “face validity”—the extent to which the emerging indicators or tools were regarded as relevant, important and interesting by the participating CSOs . As noted by Basarab Nicolescu, however, the understanding of transdisciplinarity as joint problem solving at the science-society interface represents only one of a great many potential meanings of the term [98,99,100]. Nicolescu emphasizes that looking “beyond disciplines” should not be limited to the social realm, and that complementary approaches (phenomenological, theoretical and experimental) are required if we are to understand those forms of knowledge that cut across and transcend diverse academic disciplines, as well as diverse cultures and religions . Nicolescu’s own theoretical insights, for example, include three axioms of transdisciplinarity supported by evidence from quantum physics: (i) the ontological axiom concerning the existence of different levels of Reality and different levels of perception; (ii) the logical axiom concerning the “included middle”, i.e., the possibility of being simultaneously A and non-A; and (iii) the complexity axiom of universal interdependence [99,100].
As the establishment of transdisciplinary approaches (in the broadest sense) to knowledge and education may arguably constitute a crucial element required for transitions to sustainability [7,98], the creation of Sustainable Development Goals could be a highly appropriate arena in which to examine the relevance of such ideas. In particular, an exploration of the topic of values-based indicators in relation to emerging theoretical understandings of transdisciplinarity  may be highly informative, and we propose that it could offer potential for a profound transformation of the landscape of sustainability and indicators, although such an analysis is beyond the scope of this paper.
5. Conclusions and Recommendations
In this paper we have illustrated that it is theoretically and practically possible to assess processes and outcomes relating to the less tangible dimension, consisting of human values, ethics and worldviews, which we have chosen to conceptualize as an element of the fourth pillar of sustainability (while recognizing that alternative metaphors are also plausible). We thus strongly recommend that the institutions tasked with developing Sustainable Development Goals, targets and indicators should take time, especially at this critical juncture, to reframe the sustainability assessment process and incorporate an explicit acknowledgement of its ethical dimensions.
In particular, we recommend the following policy actions:
Establishing a manageable but meaningful consultation process with key stakeholder groups within the institutions tasked with developing SDGs, to ensure that appropriate indicators and assessment tools relating to ethical values (as a key element of the “missing pillar” of sustainability) are formulated in parallel with the goals themselves;
Ensuring that projects initiated in support of SDGs are context-relevant and defined on the basis of consultation about local needs, priorities and values, rather than by the desire to improve national or global performance in relation to measurable indicators such as GDP;
Facilitating or encouraging a funding mechanism to support rigorous research into indicators and assessment methodologies focusing on sustainability processes and outcomes that are less tangible, or more qualitative in nature, than those currently being measured, including studies of perverse effects and how they might be mitigated or avoided;
Using values-based indicators to reflect on some of the complex barriers to success in achieving the Millennium Development Goals (e.g., reframing conflict as a failure to enact values such as equality and tolerance, overexploitation of finite environmental resources as a deficiency in “respect and care for the community of life”, or systemic corruption as a lack of trustworthiness and integrity), in order to identify timely measures that might be taken to address these issues from a values perspective;
Investing in research that addresses the issue of sustainability assessment in general, and values-based indicators in particular, through the lens of transdisciplinarity.
The ESDinds Project (ESDinds: The Development of Indicators and Assessment Tools for Civil Society Organizations Promoting Education for Sustainable Development, Grant Agreement No. 212237) was funded by the European Union Seventh Framework Programme, FP7/2007-2013. The authors acknowledge the valuable input of the full ESDinds project consortium, and special thanks are due to the partner CSOs: the Alliance of Religions and Conservation (ARC), Earth Charter International (ECI), European Baha’i Business Forum (ebbf) and People’s Theater. The cartoon on page 6 was created by the first author using Microsoft Office royalty-free clip art images (“Screen Beans” series)  and modified using Microsoft Paint version 6.1 together with freehand drawing.
Conflict of Interest
The authors declare no conflict of interest.
References and Notes
- Littig, B.; Griessler, E. Social sustainability: A catchword between political pragmatism and social theory. Int. J. Sustain. Dev. 2005, 8, 65–79, doi:10.1504/IJSD.2005.007375.
- Dahl, A.L. Achievements and gaps in indicators for sustainability. Ecol. Indic. 2012, 17, 14–19, doi:10.1016/j.ecolind.2011.04.032.
- Hawkes, J. The Fourth Pillar of Sustainability: Culture’s Essential Role in Public Planning; Common Ground Publishing Pty Ltd in association with the Cultural Development Network (Vic): Victoria, Australia, 2001.
- UNESCO. Mexico City Declaration on Cultural Policies. Available online: http://portal.unesco.org/culture/en/ev.php-URL_ID=12762&URL_DO=DO_TOPIC&URL_SECTION=201.html (accessed on 20 May 2013).
- UNESCO. Culture in the post-2015 sustainable development agenda: Why culture is key to sustainable development. Available online: http://www.unesco.org/new/en/culture/themes/culture-and-development/hangzhou-congress/ (accessed on 20 May 2013).
- De Leo, J. Quality Education for Sustainable Development: An Educator Handbook for Integrating Values, Knowledge, Skills and Quality Features of EDUCATION for Sustainable Development in Schooling; UNESCO APNIEVE Australia Publishing: Adelaide, Australia, 2012.
- Kagan, S. Art and Sustainability: Connecting Patterns for a Culture of Complexity. Transcript Verlag: Bielefeld, Germany, 2011.
- Woodley, E.; Crowley, E.; Dookie, C.; Carmen, E. Cultural Indicators of Indigenous Peoples’ Food and Agro-Ecological Systems; Food and Agriculture Organization of the United Nations: Rome, Italy, 2006.
- Nurse, K. Culture as the Fourth Pillar of Sustainable Development; Commonwealth Secretariat: London, UK, 2006.
- UN-PFII. Permanent Forum on Indigenous Issues Fifth Session. Available online: http://www.fao.org/docrep/009/ag253e/AG253E03.htm (accessed on 20 May 2013).
- Barkin, D.; Lemus, B. Understandig progress: a heterodox approach. Sustainability 2012, 5, 417–431, doi:10.3390/su5020417.
- Pfahl, S. Institutional sustainability. Int. J. Sustain. Dev. 2005, 8, 80–96, doi:10.1504/IJSD.2005.007376.
- Spangenberg, J.H. Institutional sustainability indicators: An analysis of the institutions in Agenda 21 and a draft set of indicators for monitoring their effectivity. Sustain. Dev. 2002, 10, 103–115, doi:10.1002/sd.184.
- Spangenberg, J.H.; Pfahl, S.; Deller, K. Towards indicators for institutional sustainability: Lessons from an analysis of Agenda 21. Ecol. Indic. 2002, 2, 61–77, doi:10.1016/S1470-160X(02)00050-X.
- World Commission on Environment and Development. Towards common action: Proposals for institutional and legal change. Our Common Future: Report of the World Commission on Environment and Development; United Nations: Geneva, Switzerland, 1987.
- Sustainable development indicators and national accounts. Available online: www.insee.fr/en/insee-statistique-publique/colloques/acn/pdf10/ravets.pdf (accessed on 20 May 2013).
- UN Division for Sustainable Development. UN Department of Economic and Social Affairs Indicators of Sustainable Development, Framework and Core Set; United Nations: New York, NY, USA, 2000.
- UN Division for Sustainable Development. Department of Policy Co-ordination and Sustainable Development Indicators of Sustainable Development, Framework and Methodologies; United Nations: New York, NY, USA, 1996.
- UN Division for Sustainable Development. Department of Policy Co-ordination and Sustainable Development Work Programme on Indicators of Sustainable Development; United Nations: New York, NY, USA, 1995.
- Clugston, R. Ethical framework for a sustainable world: Earth Charter Plus 10 conference and follow up. J. Educ. Sustain. Dev. 2011, 5, 173–176, doi:10.1177/097340821100500207.
- ECI Secretariat. Earth Charter Initiative Handbook; Earth Charter International Secretariat: San José, Costa Rica, 2010.
- Hedlund-de Witt, A. The rising culture and worldview of contemporary spirituality: A sociological study of potentials and pitfalls for sustainable development. Ecol. Econ. 2011, 70, 1057–1065, doi:10.1016/j.ecolecon.2011.01.020.
- Southern African Faith Communities Environment Institute. Interreligious Statement Towards Rio + 20 and Beyond-A Turning Point in Earth’s History; Southern African Faith Communities’ Environment Institute: Kalk Bay, South Africa, 2012.
- Oxford English Dictionary, Definition of “values”; Oxford University Press: Oxford, UK, 2013.
- Bahm, A.J. Ethics: The Science of Oughtness; Rodopi B.V.: Amsterdam, The Netherlands, 1994.
- Berg, P.G.; Nycander, G. Sustainable neighbourhoods—a qualitative model for resource management in communities. Landscape Urban Plan. 1997, 39, 117–135, doi:10.1016/S0169-2046(97)00050-9.
- Lele, S.; Norgaard, R.B. Sustainability and the scientist’s burden. Conserv. Biol. 1996, 10, 354–365.
- Bell, S.; Morse, S. Sustainability Indicators: Measuring the Immeasurable?; Earthscan: London, UK, 2008.
- ECI Secretariat. Read the Charter; Earth Charter International Secretariat: San José, Costa Rica, 2000.
- United Nations General Assembly. United Nations Millennium Declaration; United Nations General Assembly: New York, NY, USA, 2000.
- UNESCO. International Implementation Scheme for the Decade of Education for Sustainable Development; UNESCO: Paris, France, 2004.
- United Nations. Report of the World Summit on Sustainable Development; United Nations: New York, NY, USA, 2002.
- United Nations. The Future We Want: Outcome Document of the United Nations Conference on Sustainable Development in Rio de Janeiro from 20–22 June 2012; United Nations: New York, NY, USA, 2012.
- Koroneos, C.J.; Rokos, D. Sustainable and integrated development-a critical analysis. Sustainability 2012, 4, 141–153, doi:10.3390/su4010141.
- Patton, M.Q. Process use as a usefulism. New Dir. Eval. 2007, 2007, 99–112, doi:10.1002/ev.246.
- UN Department of Economic and Social Affairs, Division for Sustainable Development. Testing the CSD Indicators of Sustainable Development, Interim Analysis: Testing Process, Indicators and Methodology Sheets; UNDESA: New York, NY, USA, 1999.
- McCool, S.; Stankey, G. Indicators of sustainability: Challenges and opportunities at the interface of science and policy. Environ. Manag. 2004, 33, 294–305, doi:10.1007/s00267-003-0084-4.
- Hitlin, S.; Piliavin, J.A. Values: Reviving a dormant concept. Annu. Rev. Sociol. 2004, 30, 359–393, doi:10.1146/annurev.soc.30.012703.110640.
- Redclift, M.; Benton, T. Introduction. In Social Theory and the Global Environment; Routledge: London, UK, 1994.
- Goleman, D. Working with Emotional Intelligence; Bloomsbury: London, UK, 1998.
- Meglino, B.M.; Ravlin, E.C. Individual values in organizations: Concepts, controversies, and research. J. Manag. 1998, 24, 351–389.
- Schlater, J.D.; Sontag, M. Toward the measurement of human values. Fam. Consum. Sci. Res. J. 1994, 23, 4–25, doi:10.1177/1077727X940231002.
- Stapleton, L.M.; Garrod, G. Policy preceding possibility? Examining headline composite sustainability indicators in the United Kingdom. Soc. Indic. Res. 2008, 87, 495–502, doi:10.1007/s11205-007-9159-6.
- Grainger, A. Forest sustainability indicator systems as procedural policy tools in global environmental governance. Glob. Environ. Change 2012, 22, 147–160, doi:10.1016/j.gloenvcha.2011.09.001.
- Gudmundsson, H. The policy use of environmental indicators-learning from evaluation research. J. Transdiscipl. Environ. Stud. 2003, 2, 1–12.
- Rosenström, U. Exploring the policy use of sustainable development indicators: Interviews with Finnish politicians. J. Transdiscipl. Environ. Stud. 2006, 5, 1–13.
- Baha’i International Community. Valuing Spirituality in Development: Initial Considerations Regarding the Creation of Spiritually Based Indicators for Development; Baha’i Publishing Trust: London, UK, 1998.
- Meadows, D. Indicators and Information Systems for Sustainable Development: Report to the Balaton Group; The Sustainability Institute: Hartland Four Corners, VT, USA, 1998.
- Bell, S.; Morse, S. Sustainable development indicators: The tyranny of methodology revisited. Cons. J. Sustain. Dev. 2011, 6, 222–239.
- Rametsteiner, E.; Puelzl, H.; Alkan-Olsson, J.; Frederiksen, P. Sustainability indicator development: Science or political negotiation? Ecol. Indic. 2011, 11, 61–70, doi:10.1016/j.ecolind.2009.06.009.
- Rokeach, M. The Nature of Human Values; Free Press: New York, NY, USA, 1973.
- Rokeach, M. Understanding Human Values; Free Press: New York, NY, USA, 1979.
- Schwartz, S.H. Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries. Adv. Exp. Soc. Psychol. 1992, 25, 1–65, doi:10.1016/S0065-2601(08)60281-6.
- Schwartz, S.H. Are there universal aspects in the structure and content of human values? J. Soc. Issues 1994, 50, 19–45, doi:10.1111/j.1540-4560.1994.tb01196.x.
- Schwartz, S.H. Basic Human Values: Theory, Methods, and Applications. Available online: http://www.francoangeli.it/Riviste/Scheda_Rivista.aspx?idArticolo=30705 (accessed on 20 May 2013).
- Schwartz, S.H.; Bilsky, W. Toward a universal psychological structure of human values. J. Personal. Soc. Psychol. 1987, 53, 550–562, doi:10.1037/0022-35220.127.116.110.
- Schwartz, S.H.; Melech, G.; Lehmann, A.; Burgess, S.; Harris, M.; Owens, V. Extending the cross-cultural validity of the theory of basic human values with a different method of measurement. J. Cross-Cult. Psychol. 2001, 32, 519–542, doi:10.1177/0022022101032005001.
- Allport, G.W.; Vernon, P.E.; Lindzey, G. A Study of Values; Houghton Mifflin: Boston, MA, 1951; p. 16.
- Peterson, C.; Park, N.; Seligman, M.E.P. Assessment of character strengths. In Psychologists’ Desk Reference, 2nd; Koocher, G.P., Norcross, J.C., Hill, S.S.I., Eds.; Oxford University Press: New York, NY, USA, 2005; pp. 93–98.
- AMA. American Management Association 2002 Corporate Values Survey; AMA: New York, NY, USA, 2002.
- Davidov, E. Testing for comparability of human values across countries and time with the third round of the European Social Survey. Int. J. Comp. Soc. 2010, 51, 171–191, doi:10.1177/0020715210363534.
- Davidov, E.; Schmidt, P.; Schwartz, S.H. Bringing values back in: Testing the adequacy of the european social survey to measure values in 20 countries. Public Opin. Q. 2008, 72, 420–445, doi:10.1093/poq/nfn035.
- House, R.J.; Hanges, P.J.; Javidan, M.; Dorfman, P.W.; Gupta, V. Culture, Leadership and Organizations: The GLOBE Study of 62 Societies; Sage Publications: Thousand Oaks, CA, USA, 2004.
- Inglehart, R.; Baker, W.E. Modernization, cultural change, and the persistence of traditional values. Am. Sociol. Rev. 2000, 65, 19–51, doi:10.2307/2657288.
- Gruys, M.L.; Stewart, S.M.; Goodstein, J.; Bing, M.N.; Wicks, A.C. Values enactment in organizations: A multi-level examination. J. Manag. 2008, 34, 806–843.
- Argyris, C.; Schon, D.A. Organizational Learning; Addison-Wesley: Reading, MA, USA, 1978.
- Lahire, B. From the habitus to an individual heritage of dispositions. Towards a sociology at the level of the individual. Poetics 2003, 31, 329–355, doi:10.1016/j.poetic.2003.08.002.
- Bourdieu, P. Distinction: A Social Critique of the Judgement of Taste; Routledge: London, UK, 1984.
- Truth or Deception. Available online: http://www.thwink.org/sustain/publications/pamphlets/02_TruthOrDeception/TruthOrDeception.pdf (accessed on 20 May 2013).
- Hinkel, J. Indicators of vulnerability and adaptive capacity: Towards a clarification of the science–policy interface. Glob. Environ. Change 2011, 21, 198–208, doi:10.1016/j.gloenvcha.2010.08.002.
- Babbie, E.R. The Practice of Social Research, 12th ed.; Thomson/Wadsworth: Belmont, CA, USA, 2010.
- Peng, K.P.; Nisbett, R.E.; Wong, N.Y.C. Validity problems comparing values across cultures and possible solutions. Psychol. Methods 1997, 2, 329–344, doi:10.1037/1082-989X.2.4.329.
- Torpe, L.; Lolle, H. Identifying social trust in cross-country analysis: Do we really measure the same? Soc. Indic. Res. 2011, 103, 481–500, doi:10.1007/s11205-010-9713-5.
- Machicado, C.A.; Davis, H. A values assessment application among Bolivian managers. J. Soc. Psychol. 1988, 128, 691–693, doi:10.1080/00224545.1988.9922923.
- Cha, S.E.; Edmondson, A.C. When values backfire: Leadership, attribution, and disenchantment in a values-driven organization. Leadersh. Q. 2006, 17, 57–78, doi:10.1016/j.leaqua.2005.10.006.
- Lencioni, P.M. Make your values mean something. Harv. Bus. Rev. 2002, 80, 113–117.
- Crossley, N. Intersubjectivity: The Fabric of Social Becoming; Sage Publications: London, UK, 1996.
- Talamo, A.; Pozzi, S. The tension between dialogicality and interobjectivity in cooperative activities. Cult. Psychol. 2011, 17, 302–318, doi:10.1177/1354067X11408131.
- Podger, D.; Piggot, G.; Zahradnik, M.; Janousková, S.; Velasco, I.; Hak, T.; Dahl, A.; Jimenez, A.; Harder, M.K. The Earth Charter and the ESDinds initiative: Developing indicators and assessment tools for civil society organizations to examine the values dimensions of sustainability projects. J. Educ. Sustain. Dev. 2010, 4, 297–305, doi:10.1177/097340821000400219.
- Burford, G.; Velasco, I.; Janousková, S.; Zahradnik, M.; Hak, T.; Podger, D.; Piggot, G.; Harder, M.K. Field trials of a novel toolkit for evaluating “intangible” values-related dimensions of projects. Eval. Prog. Plan. 2013, 36, 1–14, doi:10.1016/j.evalprogplan.2012.04.005.
- Podger, D.; Velasco, I.; Luna, C.A.; Burford, G.; Harder, M.K. Can values be measured? Significant contributions from a small civil society organization through action research. Action Res. 2013, 11, 8–30, doi:10.1177/1476750312467833.
- Harder, M.K.; Burford, G.; Hoover, E. What is participation? Design leads the way to a cross-disciplinary framework. Design Issues 2013, 29. in press.
- ESDinds. Available online: http://www.esdinds.eu (accessed on 20 May 2013).
- Reed, M.S.; Fraser, E.D.G.; Dougill, A.J. An adaptive learning process for developing and applying sustainability indicators with local communities. Ecol. Econ. 2006, 59, 406–418, doi:10.1016/j.ecolecon.2005.11.008.
- ECI Secretariat. Evaluation of e-GLO3 Earth Charter Global Learning Opportunity; Earth Charter International Secretariat: San José, Costa Rica, 2011.
- Hulme, M. Problems with making and governing global kinds of knowledge. Global Environ. Change 2010, 20, 558–564, doi:10.1016/j.gloenvcha.2010.07.005.
- Ostrom, E. Polycentric systems for coping with collective action and global environmental change. Global Environ. Change 2010, 20, 550–557, doi:10.1016/j.gloenvcha.2010.07.004.
- Reed, M.; Fraser, E.D.G.; Morse, S.; Dougill, A.J. Integrating methods for developing sustainability indicators to facilitate learning and action. Ecol. Soc. 2005, 10, 1–6.
- Reed, M.S.; Dougill, A.J.; Baker, T.R. Participatory indicator development: What can ecologists and local communities learn from each other? Ecol. Appl. 2008, 18, 1253–1269.
- Reed, M.S.; Dougill, A.J. Participatory selection process for indicators of rangeland condition in the Kalahari. Geogr. J. 2002, 168, 224–234.
- Fraser, E.D.G.; Dougill, A.J.; Mabee, W.E.; Reed, M.; McAlpine, P. Bottom up and top down: Analysis of participatory processes for sustainability indicator identification as a pathway to community empowerment and sustainable environmental management. J. Environ. Manag. 2006, 78, 114–127, doi:10.1016/j.jenvman.2005.04.009.
- Krank, S.; Wallbaum, H.; Grêt-Regamey, A. Perceived contribution of indicator systems to sustainable development in developing countries. Sust. Dev. 2013, 21, 18–29.
- Arnold, H.J.; Feldman, D.C. Social desirability response bias in self-report choice situations. Acad. Manage. J. 1981, 24, 377–385, doi:10.2307/255848.
- Fisher, R.J.; Katz, J.E. Social‐desirability bias and the validity of self‐reported values. Psychol. Market. 2000, 17, 105–120, doi:10.1002/(SICI)1520-6793(200002)17:2<105::AID-MAR3>3.0.CO;2-9.
- Conrad, J. Limits to interdisciplinarity in problem oriented social science research. J. Transdiscipl. Environ. Stud. 2002, 1, 1–15.
- Burger, P.; Kamber, R. Cognitive integration in transdisciplinary science: Knowledge as a key notion. Integr. Stud. 2003, 21, 43–73.
- Gibbons, M.; Limoges, C.; Nowotny, H.; Schwartzman, S.; Scott, P.; Trow, M. The New Production of Knowledge; SAGE: London, UK, 1994.
- Nicolescu, B. The transdisciplinary evolution of the university: Condition for sustainable development. In Bulletin Interactif du Centre International de Recherches et Études transdisciplinaires; Centre International des Recherches et Etudes Transdisciplinaires (CIRET): Paris, France, 1998.
- Nicolescu, B. Manifesto of Transdisciplinarity; State Univesity of New York (SUNY) Press: New York, NY, USA, 2002.
- Nicolescu, B. Transdisciplinarity-past, present and future. In Moving Worldviews-Reshaping Sciences, Policies and Practices for Endogenous Sustainable Development; Haverkort, B., Reijntjes, C., Eds.; COMPAS Editions: Amsterdam, The Netherlands, 2006; pp. 142–166.
- Microsoft Office. Clip art, photos and animations. Available online: http://office.microsoft.com/en-us/images/ (accessed on 20 May 2013).
© 2013 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).