Effectiveness of Incorporating the Concept of City Sustainability into Sustainability Education Programs

Abstract

This study developed a sustainability education program that incorporated the concept of city sustainability, delivered it to local university students in the Jakarta Metropolitan Area (Jabodetabek), Indonesia, and then evaluated its effectiveness using questionnaire surveys. The educational materials consisted of a case story and scenario analysis report relating to city sustainability. The case story was a fictional narrative describing sustainability issues in Jabodetabek, in which the protagonist is the head of the local urban planning bureau. The scenario analysis provided three hypothetical scenarios regarding land-use patterns with predicted values of sustainability indicators in 2050. In January 2016, 46 students from Bogor Agricultural University participated in three workshops. Participants completed questionnaire surveys before and after the workshops. The results from the ordered probit models that were based on participants’ responses to 68 items of sustainability-related attitudes and perspectives showed that their participation in the workshops enhanced participants’ environmental concerns and their intention to take pro-sustainability actions. In addition, the participants tended to have a more balanced view on sustainability issues across economic, social, and environmental dimensions. This suggested that the sustainability education program focusing on city sustainability successfully enhanced the motivation of learners to contribute toward a more sustainable future.

1. Introduction

Cities and educational programs have both been regarded as essential elements for sustainable development. In 2015, a United Nations (UN) summit adopted the report ‘Transforming our world: the 2030 Agenda for Sustainable Development’ [1]. This agenda includes numerous sustainable development goals (SDGs), also called ‘global goals’. Among the 17 SDGs, two are devoted to cities and education: Goal 11 for ‘Sustainable Cities and Communities: Make cities and human settlements inclusive, safe, resilient, and sustainable’ and Goal 4 for ‘Quality Education: Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all’. These goals regard sustainable cities and sustainability education as the themes of significant importance.

Cities have been increasingly been highlighted in terms of their role in sustainability. Based on the UN’s 2014 ‘World Urbanization Prospects’ report [2], over 70% of the global population resided in rural settlements in 1950, and less than 30% lived in urban settlements. However, the urban population increased during the subsequent 60 years and, in 2014, 54% of people worldwide resided in cities. In other words, the majority of people in the world today live in a city. Furthermore, this trend of urbanization is likely to continue in the future, and the UN expects that the global percentage of the urban population will reach 66% in 2050 [2]. Thus, people in cities and their activities cannot be ignored in the quest to achieve a sustainable human society. The cumulative impact of urban environmental problems, such as air and water pollution or the unsanitary conditions in densely populated areas, is now considered to be crucial [3]. Although the population, geographical size, or total energy consumption in a single city may be negligible from a global perspective, the combined impact of all the cities around the world is so large that they cannot be ignored in terms of global sustainability.

Additionally, there is growing attention toward sustainability education as an important means for addressing environmental issues and accomplishing sustainability. The United Nations Educational, Social, and Cultural Organization (UNESCO) refers to education as a foundation for sustainable development as well as economic growth, peace, and responsible global citizenship [4]. Sustainability education is one of the most effective tools in society for addressing future issues. The role of education in achieving sustainable development was first emphasized in the ‘Agenda 21’ action plan of the UN Conference on Environment and Development in Rio de Janeiro, Brazil in 1992. Subsequently, the importance of education was emphasized throughout the World Summit on Sustainable Development in Johannesburg, South Africa in 2002 and the World Conference on Education for Sustainable Development in Bonn, Germany in 2009 [5]. More recently (2005–2014), UNESCO carried out the global ‘Decade of Education for Sustainable Development’ (DESD) campaign. DESD aimed to “integrate the values inherent in sustainable development into all aspects of learning to encourage changes in behavior that allow for a more sustainable and just society for all” [6]. The concept of ‘Education for Sustainable Development’ (ESD) has become a key theme in this trend and gained wide acceptance [5]. SDGs are aimed at improving the knowledge and skills that are necessary for promoting sustainable development and sustainable lifestyles [7]. The ultimate aim of education is shaping human behavior; societies throughout the world establish educational systems to develop citizens who will behave in desirable ways [8]. Education is a robust tool that influences human behavior, so that people can act in an ideal way to protect the environment and achieve sustainability. In past decades, a number of studies have investigated the perception of, attitude toward, knowledge of, and actions of university/college students relating to sustainability in many countries [9,10,11,12,13,14,15,16,17,18].

The growing significance of sustainable cities implies that cities should be included in sustainability education. Traditionally, sustainability education has not been designed to include city sustainability. Current sustainability education methods can be improved by including the concept of city sustainability. For example, ESD emphasizes that all three aspects of sustainability–economy, environment, and society–should be simultaneously considered [5,6], and this principle corresponds to one of the requirements for city sustainability in terms of the ‘triple bottom line’ [19]. Moreover, some studies point out that sustainability is often misunderstood in the classroom and is regarded by students and teachers as only one among several environmental issues [20,21]. Nevertheless, Jabareen [22] concluded that the themes that are related to urban planning and community planning (such as urban forms, infrastructure, and urban governance) are commonly neglected in sustainability education.

The objective of this research was to answer the question “Does an education program for sustainability incorporating the concept of ‘city sustainability’ change concerns, attitudes, and motivation for action?” An educational module that incorporated city sustainability into sustainability education was developed. Subsequently, the module was implemented in educational workshops and assessed in terms of applicability. Lastly, a questionnaire survey was administered to workshop participants and the psychological effects of the module were analyzed by comparing the responses before and after the workshops with ordered probit models.

This paper is organized, as follows. The next section describes the methods that were used in the study. This is followed by a description of the development of the teaching materials and the design of the workshops. Subsequently, the approach to implementation of the workshops, which included preparatory workshops and the main workshops, is summarized. Next, the survey and its results are reported and the significant findings from the comparative analysis are presented. Finally, concluding remarks are provided and some further issues for clarification are identified.

2. Method

2.1. City Sustainability Index

The teaching program utilized the concept of the ‘city sustainability index’ (CSI) that was proposed by Mori and Yamashita [23]. CSI is a system of indices that evaluates the sustainability of target cities. This index was designed to be utilized for urban environmental improvement while taking four factors into consideration: triple bottom line (i.e., environment, economy, and society); direct/indirect “leakage effect” of a city on other areas (such as hinterland, rural areas, other cities, etc.); the perspective of “strong sustainability”; and unbiased evaluation of cities in developed and developing countries. City sustainability denotes “maximization of the total economic and social net benefits that a city produces, without exceeding any environmental limits and while staying within acceptable limits of socio-economic inequity” [23]. CSI was applied to 18 megacities worldwide [24]. CSI was chosen as the concept for inclusion in the educational module, because CSI provides a clear definition of city sustainability and because it considers some critical concepts (such as the triple bottom line, trade-offs between environmental and economic factors, etc.) that the current ESD does not often adequately address. Notably, a clear definition of city sustainability has rarely been provided elsewhere in the scientific literature.

2.2. Educational Program

Materials were developed in the Case Method style. Case Method teaching is an educational approach in which a teacher and students proceed through a course in a collaborative way. In Case Method teaching, a “case” is used as the educational material. In the case, a story is provided from the viewpoint of a protagonist, and students are required to address the problems that the protagonist faces. This method originated from the research method that was used to analyze court cases in law schools in the United States of America (USA). The business cases developed by the Harvard Business School are the most widely used [25,26]. Unlike traditional teaching techniques in which students uniformly receive knowledge from teachers, the Case Method approach enables students to identify essential issues and consider specific solutions by themselves through discussion with other students and teachers. Steiner and Laws [27] identified the necessary competencies for addressing complicated real-world problems: comprehension of technical terms, ability to understand and design complicated systems; ability to participate in group works with relevant stakeholders; and ability to responsibly choose and take a proper approach to address the problems. Case Method teaching is an effective way of fostering critical thinking skills [25,28]. The Case Method has been introduced into sustainability education, typically in advanced courses of undergraduate programs or in graduate programs [29,30]. Projects that require the participants to participate in group work have become more popular in university education in sustainability education [30,31,32]. The group work enables the students to gain skills that relate to collaborative work in practical problems: communication among stakeholders and consensus-building among multiple players. In our workshop, the participants were required to draw causality maps regarding city sustainability. The causality map is one of the cognitive maps that shows the causal and resultant factors that are connected with arrows [33,34]. The cognitive map is often applied as part of a family of problem-structuring methods (PSMs), such as strategic options development and analysis [35]. PSM refers to a range of participatory and interactive methods for supporting group decisions in scenarios that are characterized by ‘wicked problems’ [36]. Sustainability issues are typical wicked problems, in which the problem definition and solution depends on the values, preferences, and beliefs of stakeholders [37].

The Jakarta Metropolitan Area (Jabodetabek) in Indonesia was selected as the target case. Jabodetabek has a population of 22 million, which makes it the second largest city in the world and the largest city in the developing world as of 2007 [38]. Jabodetabek is recognized as having a significant impact in terms of sustainability due to the large population size and high growth rate [39]. Educational materials were created by incorporating various topical issues in Jabodetabek such as housing, traffic volume and traffic congestion, income inequality, flooding, and other issues [40]. This was intended to motivate the students to understand the complexity of city sustainability and to assimilate multiple viewpoints on urban sustainability issues.

Furthermore, future scenario analysis of Jabodetabek was utilized, as developed by Kato and Mimura [41]. This analysis enabled the use of a scenario-based approach to address the high uncertainty in Jakarta’s future development. The scenario-based approach allows for the evaluation of multiple development patterns and an assessment of their respective impacts [42,43].

Wiek et al. [44,45] summarized the core competencies that emerge from sustainability education: systems-thinking competence, anticipatory competence, normative competence, strategic competence, and interpersonal competence. The multiple aspects of city sustainability that were included in our educational materials are expected to enhance systems-thinking competence through a holistic analysis of complex systems across environmental, social, and economic domains, which has long been recognized as a central component for achieving sustainability literacy [46,47]. An introduction to the scenario-based approach is expected to increase the anticipatory competence [42]. The Case Method reflecting real-world issues with multi-faceted sustainability dimensions and the cognitive-map production was expected to bolster the normative and strategic competencies. This is because it attempts to resolve conflicts in the trade-offs among sustainability components and because it should motivate the students to rethink social norms and to apply a more strategic approach [48]. Additionally, the workshop approach should improve the interpersonal competency through encouraging communication among participants from different social/cultural/economic backgrounds [29,49].

3. Development of Teaching Materials

3.1. Case Story

Two case materials were prepared: the case story and future scenario report. Students were expected to read both case materials and complete an assignment before attending the workshops. The case story (

Table 1. Overall Structure of the Case Story.

Day One	Day one starts with a scene at Josef’s house. As Josef leaves for his office, issues such as traffic congestion, nature loss in the city, and urbanization are described. Once he arrives at his office, he meets “Mr Xihan”, a manager of private real estate company. Mr Xihan urges Josef to pass the bill so they can carry out their large-scale real estate projects. Moreover, Mr Xihan attempts to offer a bribe to Josef. After that, the scene moves to Josef’s house, where he talks with his wife about the family budget. His wife complains about the rising gasoline price and worries that they cannot live without a car.
Day Two	On day two, Josef meets two stakeholders: Mr Susanto, the head of the city’s Economic and Industry Bureau, and Ms Lilliana Natir, the manager from the Bekasi Planning Bureau. Mr Susanto encourages Josef to submit the bill, arguing the importance of economic development and encouraging investment. On the other hand, Ms Natir opposes to the bill, insisting that the bill might worsen traffic congestion in Jakarta. Moreover, she worries about the possible negative impact on the revenue in neighboring Bekasi city. After the meetings with these stakeholders, Josef goes to a real estate agency to view properties for sale in a suburb. Josef’s wife insists on moving to the suburbs in order to avoid floods, and she dreams of having a more western lifestyle.
Day Three	On the third day, Josef meets two stakeholders from NGOs working in Jakarta: Ms Kobayashi, who is the head of an international human rights NGO and Ms Smit, the head of the WWF Jakarta office. Ms Kobayashi opposes the bill, arguing that further development within DKI Jakarta will enhance inequality, which could lead to social instability. Ms. Smit also opposes to the bill on the basis that further development will threaten biodiversity.
After meeting the various stakeholders over the three days, Josef was presented with a future scenario report for Jabodetabek. Josef will consider this report in his decision-making.

) described the fictional daily life of a protagonist (head of the Planning Bureau of Daerah Khusus Ibukota [DKI: Special Capital Region], Jakarta) in a narrative way. The purpose of the case story was to enable students to understand sustainability problems in daily life and the viewpoints of relevant stakeholders. The case story included issues such as traffic congestion, urban sprawl, energy consumption, and floods. In addition, the story involved stakeholders: a private real estate company, a representative from a neighboring city, non-governmental organizations (NGOs), officers of the World Wildlife Fund (WWF), and others. In the story, stakeholders took actions to influence the decision-making of the protagonist. The case story began with the following abstract: “As the head of the urban planning bureau of DKI Jakarta, Josef Djuhara was pressed to make a decision for the submission of a bill that allows new construction on natural lands such as wetlands, within DKI Jakarta. If the bill is passed in parliament, it would accelerate development in DKI Jakarta. However, there has been no consensus between various stakeholders in terms of the benefits/costs and the expected results of the bill. Thus, they have pressured Josef to consider their interests. As such, Josef could not decide whether to submit the bill. The following essay illustrates a part of Josef’s daily life”. The story then described three days from Josef’s point of view.

The case story illustrated a possible situation that a person in charge of urban planning would face. The participants were expected to consider how urban planning policy can be connected to sustainability issues, what stakeholders groups would be involved, and how the stakeholders would influence the process of decision-making that would affect the long term development of a mega city.

3.2. Scenario Report

The scenario report described the sustainability of Jabodetabek in 2050 and it was based on a previous research by Kato and Mimura [41]. A scenario-based approach was employed to consider future uncertainty. The purpose of the future scenario report was to enable students to understand the possible situations concerning sustainability in Jabodetabek through the quantitative analysis of some relevant indices that are based on CSI. The definition of sustainability and the concepts of constraint indicators and maximization indicators in CSI were intentionally masked in the future scenario report to allow for students to evaluate city sustainability.

Three scenarios describing conditions in 2050 were provided. The scenarios were three forms of land-use patterns that were calculated from assumed population density distribution: the ‘monocentric scenario’, ‘spread scenario’, and ‘polycentric scenario’. To create these three scenarios, eight types of urban space categories were defined as land-use patterns: water area, natural area, four types of residential areas (urban village, rural, planned village, and high-rise apartment), commercial area, and industrial area. Subsequently, land-use patterns were determined based on the multinomial logit model analysis of population density distribution.

The monocentric scenario included a number of assumptions: the strong promotion of high value-added industries (e.g., global financing, knowledge economy); insufficient public transport infrastructure; and inappropriate land-use policy. The scenario also assumed an increase in demand for office space in the city center, forcing the manufacturing industry out to other regions. High-income residents were assumed to live in high multi-story buildings in the city center, while the number of lower-income people would increase and would live in low-rise residential buildings that are concentrated along rivers within the city. As a result, income inequality would increase and spatial income disparities within the city would become more evident.

The spread scenario assumed that the manufacturing sector would be promoted under domestic industry protection policies, thus accelerating industrial park construction. As a result, many workers would commute from various areas in the city to these industrial parks, and many low-and medium-rise residential buildings would be constructed in suburban areas. Moreover, an increase in trade would drive the economy. Serious traffic congestion would occur in the city.

The polycentric scenario assumed that policies to promote specific industries at multiple locations within the city would be introduced and that development would be restricted to these areas. Traffic congestion problems would be resolved through the implementation of an efficient public transport network that connects these development centers. In each development center, several types of industries would be encouraged and economic growth would be promoted through cross-synergies. Although income inequality would expand, the average income would increase and the standard of living environment would improve accordingly.

Furthermore, each scenario was assessed while using nine CSI indicators: CO₂ emissions from construction materials and household consumption, the Gini index, average household income, economic impact from industries, urban biodiversity, flood-affected areas and flood-affected population, and individual satisfaction with the built environment. The three scenarios were characterized by the values of the indicators. On this basis, the students were expected to consider the sustainability of Jabodetabek for the year 2050 in terms of future scenarios.

4. Workshops

4.1. Design of Workshops

Visual presentations (slide shows) were created for the educational workshops. A role-playing style was used in the workshops, in which each group of participants had to play the role of staff members working in the Planning Bureau of DKI Jakarta, and propose a future development vision for Jabodetabek, examining the three future scenarios. Furthermore, an assignment was required to be completed by students before attending the workshops, to help the students gain understanding and to enable them to sufficiently envision the future of Jakarta. All of the materials were prepared in English and the workshops were conducted in English. The workshops each consisted of six sessions.

4.1.1. Introductory Session

In the introductory session, the program and structure of the workshop were presented and the background of the participants’ roles as planning department staff members was explained.

4.1.2. First Task Session

In the second session, the participants were tasked with identifying the environmental, social, and economic problems in Jabodetabek, and then compiling a causality map that showed the causal relationships of the problems. This first task had three objectives: to enable participants to consider triple bottom line issues in the city comprehensively, to critically and creatively assess the causes and effects through individual logical thinking and group discussion, and to understand the connections among activities within cities and their impact inside and outside the city. This task highlighted two teaching points. First, sustainability issues are interconnected in terms of environmental, social, and economic aspects, and thus each issue cannot be solved in isolation, but rather require a strategic approach to devising effective solutions. Second, a city cannot exist independently and it relies on other areas to obtain resources and ecosystem services.

4.1.3. Second Task Session

In the second task in the third workshop session, the participants were asked to consider the interests and conflicting views of the relevant stakeholders based on the causality maps that were prepared in the first task. The second task had two objectives: to identify how and what groups of stakeholders are involved in sustainability issues and to consider which stakeholders must be included in negotiations or trade-offs. This task revealed the variety of possible opinions/arguments among different stakeholder groups based on differing causalities to participants. Moreover, different causalities also imply that dealing with an issue can negatively affect the interests of other stakeholders who are not directly involved in the issue.

4.1.4. Third Task Session

In the third task, the participants were asked to identify six indicators in the future scenario report and in the causality maps that participants had prepared in the first task. The example factors were CO₂ emissions (from construction materials and household consumption), economic inequality (the Gini index), economic efficiency (average household income and economic impact from industries), urban biodiversity, risk of flooding (flood-affected areas and flood-affected population), and comfort with daily life (individual satisfaction with the built environment). Afterwards, the participants were required to identify the trade-offs between these factors. This task had three objectives: to consider the causal relationships related to sustainability in the future scenario report for Jabodetabek, to identify trade-offs, and to highlight the need for negotiations and trade-offs. The task showed that decision-makers are faced with trade-offs when solving sustainability issues. Furthermore, decision-makers must decide which issue should be prioritized when facing a trade-off. Finally, the task highlighted that negotiations play a significant role in solving issues in the face of trade-offs.

4.1.5. Fourth Task Session

The final task in the workshop required each group of participants to select one of the three scenarios (monocentric, polycentric, and spread) as an ideal vision for future Jabodetabek. The participants were also asked to give presentations to the workshop coordinator who played a role as the head of the urban planning bureau. In the presentations, participants described the selected scenario and explained the reason for selection. The objective of this task was to encourage participants to select a vision for the ideal future development of Jabodetabek based on legitimate reasons. After group discussion, each group gave a presentation to all participants and explained which scenario they chose and for what reason. The coordinator and other participants interrogated the presenters. After all of the presentations were delivered, the coordinator revealed the sustainable scenario that was reported by Kato et al. [50] and described the criteria that supported this answer. Furthermore, the coordinator explained that the workshop was designed based on the concepts of the CSI.

4.1.6. Wrap-Up Session

In the last session of the workshop, the concepts of the CSI were explained in detail. First, the coordinator explained the definitions of city sustainability. Subsequently, a 5-min. introductory movie on CSI was shown, which was created by the Megacities and the Global Environment Project in the Research Institute for Humanity and Nature, Japan. After showing this film, the coordinator summarized the objectives of the workshop.

4.1.7. Implementation of Workshops

The workshops were conducted to evaluate the applicability of the materials and the teaching method. First, preparatory workshops were held at two Japanese universities (Shiga University and the University of Tokyo). Based on feedback obtained at these preparatory workshops, the teaching materials were modified and finalized for use in the main workshop series to be presented in Indonesia. The main workshop was held over three days at Bogor Agricultural University in Indonesia on 11–13 January 2016.

4.2. Main Workshop in Indonesia

A total of 46 students participated in the three main workshops (15 students each in the first two workshops and 16 in the third). Each student was allowed to participate in only one workshop. The participants were recruited through a Facebook post that was shared by a project collaborator at Bogor Agricultural University. Of the 56 applicants, 46 students were selected on a first-come-first-served basis. A pre-workshop questionnaire was distributed by email to the confirmed participants, which was completed and then returned to the collaborator. The participants were enrolled in bachelor’s, master’s, and doctorate courses that included 10 majors (Management of Land Resources = 14; Landscape Architecture = 15; Agronomy and Horticulture = 4; Regional Planning Science = 1; Nutrition Science = 2; Agro-industrial Technology = 2; Marine and Coastal Resources Management = 1; Silviculture = 3; and, Forestry = 2). All 46 participants indicated that they had previously received environmental education and 44 participants said that they had received sustainability education.

Table 2. Descriptive Statistics of Workshop (WS) Participants.

	Mean	S.D.	Min.	Median	Max.
Participation of WS (1: Yes; 0: No)	0.500	0.052	0	0.5	1
Member of WS1 (1: Yes; 0: No)	0.326	0.049	0	0	1
Member of WS2 (1: Yes; 0: No)	0.326	0.049	0	0	1
Age	23.20	0.255	20	22	29
Gender (1: Male; 0: Female)	0.348	0.050	0	0	1
Origin from Jabodetabek (1: Yes; 0: No)	0.283	0.047	0	0	1
Awareness about Triple bottom line (1: Yes; 0: No)	0.500	0.052	0	0.5	1
Awareness about Strong/weak Sustainability (1: Yes; 0: No)	0.587	0.052	0	1	1

summarizes the descriptive statistics of the participants.

Workshop materials (case story and future scenario report) and an assignment were sent to the participants after the completed pre-workshop questionnaire was returned to the coordinator. Completed assignments had to be returned to the coordinator by email before the workshops began. Each workshop was presented from 9 a.m. to 3 p.m., including short breaks and a one-hour lunch break. The participants in each workshop were divided into three groups that achieved a balance in the mix of the participants with different social/cultural/economic backgrounds. The workshop was conducted in English, but there was no restriction on using local language during the group discussions. Before leaving each workshop, the participants completed a post-workshop questionnaire.

5. Pre- and Post-Workshop Surveys

A questionnaire survey was developed and administered to evaluate participants’ values and attitudes toward sustainability before and after a workshop; the same questionnaire was used both times for each of the three workshops. A five-point Likert-scale was used for responses (5: strongly agree; 4: agree; 3: neither agree nor disagree; 2: disagree; 1: strongly disagree). The questionnaire focused on five key areas: city sustainability, a new ecological paradigm, sustainability concerns, responsible stakeholders, and pro-sustainability behavior. Appendix A lists the 64 question items answered by workshop participants.

5.1. City Sustainability Index

To evaluate each student’s awareness of city sustainability according to the CSI, 17 questions were included in the questionnaire, concerning the constraint indicators of environmental aspects (CSI. 1–CSI. 4), constraint indicators of social and economic aspects (CSI. 5–CSI. 7), maximization indicators (CSI. 8–CSI. 12), human sustainability (CSI. 13 and CSI. 14), and relationships between human sustainability and city sustainability (CSI. 15–CSI. 17).

5.2. New Ecological Paradigm Scale

The New Ecological Paradigm (NEP) scale [51] was utilized for the survey and it comprised 15 items that explored the relationships between humans and the environment. The first version of NEP was published in 1978, and it has been widely used for evaluating people’s attitudes toward the environment [52,53,54]. Moreover, the NEP is commonly used for evaluating the effects of environmental education [55]. The revised NEP scale that was released in 2000 [56] was utilized in the workshop questionnaire and it assessed the knowledge of respondents in relation to the limits to growth (NEP. 1, NEP. 6, and NEP. 11), anti-anthropocentrism (NEP. 2, NEP. 7, and NEP. 12), the fragility of nature’s balance (NEP. 3, NEP. 8, and NEP. 13), rejection of exceptionalism (NEP. 4, NEP. 9, and NEP. 14), and the possibility of an eco-crisis (NEP. 5, NEP. 10, and NEP. 15). The NEP scale gives a concise but comprehensive measurement of a respondent’s attitudes toward the environment.

5.3. Concerns about Sustainability Issues

Measuring concerns about environmental problems is important in evaluating the effectiveness of an educational module. According to the norm activation theory of Schwartz [57], environmental concerns and awareness of consequences are assumed to activate people’s behavior in a pro-environmental way. Furthermore, adverse consequences of environmental problems are assumed to activate people’s intentions of adopting pro-environmental behavior, according to the value belief norm theory, which is an extension of norm activation theory [58,59].

Questions regarding concern for the environment (CON. 1–CON. 13 and CON. 22–CON. 25) were based on Schultz [53]. They included at least four items in each of three categories of concern: concern about the biosphere (plants, marine life, birds, and animals); concern about oneself (me, my lifestyle, my health, and my future); and, concern about others (people in my city, people in my county, all people in the world, children, and future generation). The ‘people in my city’ was added to the original 12 items of Schultz [53] to account for the urban perspective. In addition, four items were added for measuring the degree of general concern about the ‘future of my city’, ‘future of my country’, ‘future of human society’, and ‘future of the Earth’. The 17 questions coincided with the 17 items to quantify the respondents’ concerns. In addition, six more questions (CON. 26–CON. 31) were included to measure six additional items that identified specific concerns of respondents. Two items in relation to economic problems were ‘relative poverty’ and ‘income inequality’; two items about social problems were ‘lack of basic human needs’ (e.g., health, safe drinking water, and education) and ‘inequalities in access to basic human needs’; and, two items about environmental problems were ‘loss of biodiversity’ and ‘global warming’.

5.4. Responsible Stakeholders

A feeling of obligation links perceived need with the actual actions of individuals [57]. Thus, eight questions (CON. 14–CON. 21) were constructed to measure the respondent’s relative intensity of feeling of obligation regarding different subjects. The questions probed feelings on subjects that ranged from personal and private to general and large-scale.

5.5. Pro-Sustainability Attitudes

It is necessary to assess participants’ pro-sustainability behavior because sustainability education aims to shape people’s behavior in a more sustainable way. Thus, the questionnaire examined respondents’ intention of taking pro-sustainability or/and pro-environmental actions, given that the intention behind the actions is considered to be a crucial determinant in activating actual behavior. Although other factors than intention behind the actions, such as social and institutional factors, could also affect pro-sustainability behavior, they were not included in our questionnaire survey. This is one of our further issues. The behavior intention scale items of Stern et al. [60] were adopted for the questionnaire. The scales have been shown to be useful for measuring willingness to act for solutions and thus have been widely used in research on environmental psychology [52,61]. In the questionnaire, five items (BHV. 1–BHV. 5, see Appendix A for details) were included to measure the willingness to take an action toward sustainability and the respondent’s attitude. Items 1, 3, and 4 were based on Stern et al. [60]. Items 2 and 5 were specifically created for the questionnaire, but they were derived from Stern et al. [60]. Items 3 and 4 measured the intention of taking actions toward solutions to environmental issues, and item 1 measured respondents’ attitude to the consequence of environmental actions. Items 2 and 5 measured the respondents’ intention behind taking actions toward economic inequality and poverty, respectively.

6. Results

6.1. Findings

The effectiveness of the workshops and teaching materials were examined while using the participants’ responses to the completed pre- and post-workshop questionnaires for the three workshops (WS1, WS2, and WS3) in Bogor Agricultural University. Appendix B summarizes the means of the responses for all items before and after the three workshops. Although the changes in responses between before and after the workshops could be directly examined using a simple statistical test such as Wilcoxon’s test, these may be biased because the responses in our survey are constrained between 1 to 5. For instance, if the mean of responses would be nearly 5 before the workshop, the positive effect of the workshop would not be evident. Subsequently, ordered probit models are employed, in which the independent variables are: a dummy of participation in the workshop (1 if a respondent participates in the workshop and 0 otherwise); a dummy of a member in WS1 (1 if a respondent participates in WS1 and 0 otherwise); a dummy of a member in WS2 (1 if a respondent participates in WS2 and 0 otherwise); age of respondent; a dummy of gender (1 if a respondent is male and 0 otherwise); a dummy of origin of respondent (1 if the origin of the respondent is Jabodetabek and 0 otherwise); a dummy of awareness about triple bottom line (1 if a respondent is aware of triple bottom line and 0 otherwise); a dummy of awareness about strong/weak sustainability (1 if a respondent is aware of strong/weak sustainability and 0 otherwise). The estimation results are summarized in

Table 3. Estimation Results of Ordered Probit Models—City Sustainability Index (CSI).

	CSI. 1		CSI. 2		CSI. 3		CSI. 4		CSI. 5		CSI. 6		CSI. 7		CSI. 8		CSI. 9
Variables	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE
Intercept	1.14	1.4	1.51	1.3	3.58 ***	1.2	−0.96	1.3	5.03 ***	1.3	1.84	1.2	5.20 ***	1.3	3.53 ***	1.2	0.31	1.2
WS	−0.53 **	0.3	0.57 **	0.2	−0.08	0.2	0.43 *	0.2	−0.32	0.2	0.63 ***	0.2	0.08	0.2	−0.30	0.2	−0.29	0.2
WS1	0.12	0.4	0.43	0.3	−0.46	0.3	0.77 **	0.3	−0.98 ***	0.3	0.25	0.3	−0.15	0.3	−0.25	0.3	0.21	0.3
WS2	0.64 *	0.3	−0.04	0.3	−1.24 ***	0.3	−0.78 ***	0.3	−0.38	0.3	−0.56 *	0.3	−0.30	0.3	−0.52 *	0.3	−0.83 ***	0.3
Age	−0.04	0.1	0.03	0.1	−0.01	0.1	0.13 **	0.1	−0.17 ***	0.1	−0.00	0.1	−0.19 ***	0.1	−0.04	0.1	0.10 *	0.1
Gender	−0.94 ***	0.3	0.78 ***	0.3	0.82 ***	0.3	0.81 ***	0.3	−0.97 ***	0.3	0.09	0.3	−1.09 ***	0.3	−0.03	0.3	0.01	0.3
Jabodetabek	−0.76 **	0.3	−0.11	0.3	0.05	0.3	0.91 ***	0.3	−0.44	0.3	−0.25	0.3	−0.52 *	0.3	0.02	0.3	0.13	0.3
TBL	0.40	0.3	−0.47 *	0.3	−0.47*	0.3	−0.27	0.3	0.44	0.3	0.00	0.3	0.76 ***	0.3	−0.23	0.3	−0.76 ***	0.3
SWSUS	0.07	0.3	−0.10	0.3	−0.12	0.3	−0.06	0.3	0.07	0.3	−0.09	0.3	−0.13	0.3	−0.03	0.3	0.75 ***	0.3
Threshold 1/2	1.69 ***	0.2	0.96 ***	0.2	0.65 ***	0.2	0.82 ***	0.2	1.56 ***	0.2	1.10 ***	0.2	1.58 ***	0.2	1.35 ***	0.2	1.36 ***	0.2
Threshold 2/3	1.93 ***	0.3	1.34 ***	0.2	1.95 ***	0.2	2.01 ***	0.2	1.81 ***	0.2	2.45 ***	0.2	2.04 ***	0.2	2.12 ***	0.1	2.44 ***	0.2
Threshold 3/4	n.a.	n.a.	3.00 ***	0.2	3.55 ***	0.2	3.72 ***	0.2	2.89 ***	0.4	4.12 ***	0.3	2.82 ***	0.3	3.80 ***	0.2	4.69 ***	0.4
Final LL	−73.1		−94.2		−95.1		−96.8		−90.9		−106.7		−92.9		−114.5		−99.4
Initial LL	−84.7		−102.6		−110.8		−117		−103.3		−115.3		−108.8		−117.8		−111
Chi-squared	23.1		16.7		31.4		40.3		24.6		17.2		31.7		6.7		23.1
Significance	0.003		0.033		0.000		0.000		0.001		0.027		0.000		0.559		0.003
Pseudo R2	0.136		0.081		0.141		0.172		0.119		0.074		0.146		0.028		0.104
N	92		92		92		92		92		92		92		92		92
	CSI. 10		CSI. 11		CSI. 12		CSI. 13		CSI. 14		CSI. 15		CSI. 16		CSI. 17
Variables	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE
Intercept	−0.75	1.2	0.39	1.3	1.23	1.2	5.00 ***	1.4	−1.51	1.3	4.50 ***	1.2	0.91	1.4	−0.62	1.5
WS	0.2	0.2	−1.08 ***	0.3	−0.08	0.2	−0.59 **	0.3	−0.10	0.2	−0.17	0.2	−0.15	0.3	−0.09	0.3
WS1	0.61 **	0.3	0.29	0.3	−0.15	0.3	−0.63 *	0.3	0.15	0.3	0.19	0.3	0.38	0.4	0.38	0.4
WS2	0.13	0.3	0.33	0.3	−0.22	0.3	−0.16	0.3	0.2	0.3	−0.44	0.3	−0.34	0.3	−0.24	0.3
Age	0.06	0.1	0.02	0.1	0.03	0.1	−0.15 ***	0.1	0.13 **	0.1	−0.08 *	0.1	0.07	0.1	0.13 **	0.1
Gender	0.24	0.3	−0.34	0.3	0.62 **	0.3	−0.42	0.3	−0.05	0.3	0.41	0.3	0.85 ***	0.3	0.76 **	0.3
Jabodetabek	−0.09	0.3	−0.43	0.3	0.08	0.3	−0.45	0.3	−0.36	0.3	0.27	0.3	−0.05	0.3	0.08	0.3
TBL	0.06	0.3	0.07	0.3	−0.33	0.3	−0.32	0.3	−0.55 *	0.3	−0.52 *	0.3	0.3	0.3	0.03	0.3
SWSUS	−0.26	0.3	−0.36	0.3	0.15	0.3	0.51 *	0.3	0.25	0.3	0.23	0.3	−0.44	0.3	−0.44	0.3
Threshold 1/2	0.83 ***	0.1	1.73 ***	0.2	0.89 ***	0.2	2.12 ***	0.2	1.98 ***	0.2	1.01 ***	0.2	0.69 ***	0.2	0.26	0.2
Threshold 2/3	2.39 ***	0.2	2.00 ***	0.2	1.36 ***	0.1	3.39 ***	0.4	n.a.	n.a.	1.84 ***	0.2	2.32 ***	0.3	0.84 ***	0.2
Threshold 3/4	n.a.	n.a.	n.a.	n.a.	2.80 ***	0.2	n.a.	n.a.	n.a.	n.a.	3.44 ***	0.2	n.a.	n.a.	2.00 ***	0.2
Final LL	−109.1		−78.5		−116.5		−75		−76.7		−104.6		−67.4		−74.2
Initial LL	−112.8		−90.6		−121		−87		−82.2		−111.4		−75.8		−81.8
Chi-squared	7.4		24.3		8.8		24		10.8		13.5		16.6		15.1
Significance	0.488		0.001		0.353		0.002		0.209		0.093		0.034		0.056
Pseudo R2	0.033		0.134		0.036		0.137		0.066		0.06		0.109		0.092
N	92		92		92		92		91		92		92		92

Note 1: Coef: Coefficient; SE: Standard Error; WS: Participation of WS; WS1: Member of WS1; WS2: Member of WS2; TBL: Triple bottom line; SWSUS: Strong/weak Sustainability; LL: Log Likelihood; and n.a.: not available. Note 2: *: p < 0.1; **: p < 0.05; ***: p < 0.01.

,

Table 4. Estimation Results of Ordered Probit Models—New Ecological Paradigm (NEP).

	NEP. 1		NEP. 2		NEP. 3		NEP. 4		NEP. 5		NEP. 6		NEP. 7		NEP. 8
Variables	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE
Intercept	2.63 **	1.2	2.18 *	1.2	−0.13	1.3	−0.96	1.2	0.54	1.2	−1.16	1.3	4.51 ***	1.5	3.25 ***	1.2
WS	0.08	0.2	−0.14	0.2	−0.34	0.2	0.01	0.2	0.05	0.2	0.13	0.2	0.67 **	0.3	0.09	0.2
WS1	0.23	0.3	−0.74 **	0.3	0.21	0.3	0.48	0.3	0.3	0.3	0.17	0.3	−0.76 **	0.4	0.03	0.3
WS2	−1.02 ***	0.3	−0.30	0.3	−0.64 **	0.3	0.02	0.3	−0.59 **	0.3	−0.24	0.3	−1.58 ***	0.4	−0.49 *	0.3
Age	−0.07	0.1	0	0.1	0.09 *	0.1	0.13 ***	0.1	0.09 *	0.1	0.12 **	0.1	−0.04	0.1	−0.05	0.1
Gender	0.18	0.3	−0.12	0.3	0.42	0.3	0.09	0.2	0.02	0.3	1.32 ***	0.3	0.08	0.3	−0.25	0.3
Jabodetabek	0.41	0.3	−0.32	0.3	0.57 **	0.3	0.11	0.3	0.66 **	0.3	0.53*	0.3	1.11 ***	0.3	−0.36	0.3
TBL	−0.47 *	0.3	−0.20	0.3	−0.30	0.3	−0.16	0.3	−0.32	0.3	−0.31	0.3	0.07	0.3	0.3	0.3
SWSUS	0.59 **	0.3	0.3	0.3	0.1	0.3	−0.27	0.3	−0.65 **	0.3	−0.18	0.3	−0.76 **	0.3	−0.09	0.3
Threshold 1/2	0.88 ***	0.1	0.95 ***	0.1	0.99 ***	0.2	1.27 ***	0.1	1.29 ***	0.2	0.56 ***	0.2	0.93 ***	0.3	1.69 ***	0.1
Threshold 2/3	2.55 ***	0.2	1.68 ***	0.1	2.984 ***	0.2	2.13 ***	0.1	2.19 ***	0.1	2.29 ***	0.2	3.04 ***	0.3	2.25 ***	0.1
Threshold 3/4	n.a.	n.a.	3.44 ***	0.2	n.a.	n.a.	3.50 ***	0.2	3.71 ***	0.2	n.a.	n.a.	n.a.	n.a.	3.46 ***	0.2
Final LL	−101.8		−114.4		−88.7		−121		−112.4		−84.6		−63.6		−120.1
Initial LL	−114.4		−119.7		−97.1		−126.4		−125.1		−98.5		−82		−125.4
Chi-squared	25.3		10.7		16.8		10.9		25.4		27.8		36.9		10.6
Significance	0.001		0.217		0.031		0.207		0.001		0.000		0.000		0.22
Pseudo R2	0.11		0.044		0.086		0.043		0.101		0.141		0.224		0.042
N	92		92		92		92		92		92		92		92
	NEP. 9		NEP. 10		NEP. 11		NEP. 12		NEP. 13		NEP. 14		NEP. 15
Variables	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE
Intercept	0.47	1.3	4.32 ***	1.2	2.97 **	1.2	1.07	1.2	−1.96	1.4	0.99	1.3	0.5	1.2
WS	−0.23	0.2	0.23	0.2	0.2	0.2	0.34	0.2	−0.13	0.2	0.23	0.2	0.18	0.2
WS1	−0.12	0.3	0.32	0.3	0.19	0.3	−0.15	0.3	0.45	0.3	0.36	0.3	0.79 **	0.3
WS2	0.48	0.3	−0.07	0.3	−0.36	0.3	−0.59 **	0.3	−1.10 ***	0.3	−0.10	0.3	0.06	0.3
Age	0.08	0.1	−0.12 **	0.1	−0.06	0.1	−0.01	0.1	0.21 ***	0.1	0.02	0.1	0.07	0.1
Gender	0.04	0.3	−0.06	0.3	0.55 **	0.3	0.1	0.3	0.55 **	0.3	0.60 **	0.3	0.80 ***	0.3
Jabodetabek	−0.07	0.3	−0.37	0.3	0.70 ***	0.3	−0.08	0.3	0.88 ***	0.3	−0.43	0.3	0.3	0.3
TBL	0.63 **	0.3	0.12	0.3	−0.24	0.3	0.03	0.3	0.34	0.3	−0.36	0.3	−0.28	0.3
SWSUS	−0.56 **	0.3	0.08	0.3	0.25	0.3	0.19	0.3	−0.10	0.3	0.19	0.3	−0.28	0.3
Threshold 1/2	0.65 ***	0.2	1.09 ***	0.1	0.84 ***	0.2	0.82 ***	0.1	1.99 ***	0.2	0.66 ***	0.2	0.68 ***	0.2
Threshold 2/3	1.38 ***	0.2	1.71 ***	0.1	1.65 ***	0.1	1.23 ***	0.1	2.44 ***	0.2	2.83 ***	0.2	1.51 ***	0.2
Threshold 3/4	3.56 ***	0.2	3.41 ***	0.3	3.15 ***	0.2	2.93 ***	0.3	4.59 ***	0.3	n.a.	n.a.	3.17 ***	0.2
Final LL	−89.1		−119.3		−111.8		−126.3		−87.9		−82.5		−97.8
Initial LL	−95.8		−126.4		−120.2		−130.6		−112.2		−87.6		−105.7
Chi-squared	13.3		14.2		16.8		8.7		48.5		10.2		15.9
Significance	0.099		0.076		0.031		0.366		0		0.246		0.042
Pseudo R2	0.069		0.056		0.07		0.033		0.216		0.058		0.075
N	92		92		92		92		92		91		92

Note 1: Coef: Coefficient; SE: Standard Error; WS: Participation of WS; WS1: Member of WS1; WS2: Member of WS2; TBL: Triple bottom line; SWSUS: Strong/weak Sustainability; LL: Log Likelihood; and n.a.: not available. Note 2: *: p < 0.1; **: p < 0.05; ***: p < 0.01.

and

Table 5. Estimation Results of Ordered Probit Models (CON and BHV).

	CON.1		CON.2		CON.3		CON.4		CON.5		CON.6		CON.7		CON.8		CON.9		CON.10		CON.11		CON.12		CON.13		CON.14		CON.15		CON.16		CON.17		CON.18
Variables	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE
Intercept	1.66	1.5	3.29**	1.5	0.56	1.4	1.33	1.4	−1.36	1.5	2.45**	1.2	−2.06	1.9	0.22	1.8	0.90	1.4	1.73	1.4	0.31	1.4	−0.47	1.7	2.27	409,383.1	−3.64**	1.8	1.92	1.4	3.60**	1.5	0.06	1.3	−0.58	1.7
WS	0.00	0.3	−0.12	0.3	0.24	0.3	−0.10	0.3	0.62**	0.3	0.42*	0.2	1.26***	0.4	0.57*	0.3	0.49*	0.3	0.64**	0.3	0.57**	0.3	0.56*	0.3	0.80*	0.4	0.40	0.3	0.02	0.3	0.55**	0.3	0.25	0.3	0.44	0.3
WS1	−0.01	0.4	0.26	0.4	0.48	0.4	0.13	0.4	0.05	0.4	−0.36	0.3	0.11	0.5	0.32	0.5	0.50	0.4	0.20	0.4	−0.28	0.4	0.35	0.4	−14.65	409,383.1	−0.35	0.4	0.25	0.4	0.25	0.4	0.82**	0.4	0.86**	0.4
WS2	−1.47***	0.4	−1.32***	0.4	−1.04***	0.4	−1.15***	0.4	−0.15	0.4	−0.26	0.3	−0.82*	0.4	−0.59	0.4	−0.21	0.3	−0.39	0.3	−0.86**	0.4	−0.20	0.4	−16.50	409,383.1	−1.10***	0.4	−0.86**	0.3	−0.51	0.3	−0.35	0.3	−0.22	0.3
Age	−0.04	0.1	0.00	0.1	0.10*	0.1	0.06	0.1	0.04	0.1	−0.02	0.1	0.11	0.1	0.01	0.1	0.02	0.1	−0.01	0.1	0.08	0.1	0.11	0.1	0.58**	0.2	0.26***	0.1	0.00	0.1	−0.07	0.1	0.04	0.1	0.12*	0.1
Gender	0.39	0.4	0.17	0.3	0.30	0.3	−0.22	0.3	0.94***	0.3	−0.02	0.3	0.98**	0.4	0.87**	0.4	0.44	0.3	0.54*	0.3	0.46	0.3	0.17	0.3	−0.59	0.5	0.14	0.3	0.54	0.3	0.22	0.3	0.34	0.3	0.74**	0.4
Jabodetabek	0.56	0.4	0.73**	0.4	0.67**	0.3	0.33	0.3	0.82**	0.4	0.51*	0.3	0.57	0.4	0.18	0.4	0.06	0.3	0.39	0.3	0.30	0.3	0.63*	0.4	1.33**	0.6	−0.06	0.3	0.21	0.3	0.19	0.3	0.22	0.3	0.01	0.3
TBL	−0.00	0.4	−0.33	0.3	−0.32	0.3	−0.21	0.3	0.01	0.3	0.18	0.3	−0.50	0.4	−0.28	0.4	0.01	0.3	0.10	0.3	−0.19	0.3	0.12	0.3	−1.58**	0.7	0.01	0.3	−0.26	0.3	0.28	0.3	0.12	0.3	−0.28	0.3
SWSUS	−0.17	0.3	−0.43	0.3	−0.28	0.3	−0.19	0.3	−0.46	0.3	0.47*	0.3	−0.44	0.4	−0.16	0.4	−0.13	0.3	−0.21	0.3	−0.09	0.3	−0.36	0.3	2.08***	0.8	0.13	0.3	−0.26	0.3	−0.28	0.3	−0.17	0.3	−0.38	0.3
Threshold 1/2	n.a.	n.a.	2.74***	0.5	0.66**	0.3	2.05***	0.3	n.a.	n.a.	0.29	0.2	n.a.	n.a.	n.a.	n.a.	1.48***	0.2	1.87***	0.3	1.85***	0.3	1.89***	0.4	n.a.	n.a.	1.92***	0.3	1.34***	0.2	1.88***	0.3	1.19***	0.2	2.04***	0.4
Threshold 2/3	n.a.	n.a.	n.a.	n.a.	3.01***	0.3	n.a.	n.a.	n.a.	n.a.	1.67***	0.2	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.
Threshold 3/4	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	2.97***	0.2	n.a.	n.a	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.
Final LL	−47.6		−54.6		−65.6		−63.3		−51.3		−98.6		−37.9		−42.1		−71		−66.1		−64.4		−51.3		−23.1		−57.9		−67		−61.5		−76.4		−53.4
Initial LL	−60.6		−67.5		−78.2		−73		−62.1		−105		−52.5		−49.1		−76.1		−73.3		−72.7		−57.6		−40.7		−71.4		−75.2		−68.2		−84.3		−61.3
Chi-squared	25.9		25.7		25.1		19.3		21.6		12.6		29		14		10.2		14.5		16.5		12.6		35.1		26.9		16.3		13.4		15.7		15.7
Significance	0.001		0.001		0.001		0.013		0.005		0.124		0.000		0.081		0.251		0.069		0.035		0.125		0.000		0.000		0.037		0.096		0.045		0.046
Pseudo R2	0.213		0.190		0.160		0.132		0.174		0.060		0.276		0.142		0.067		0.098		0.113		0.109		0.430		0.188		0.108		0.098		0.093		0.128
N	91		91		91		91		91		91		91		91		91		91		91		91		91		91		91		91		91		91
	CON.19		CON.20		CON.21		CON.22		CON.23		CON.24		CON.25		CON.26		CON.27		CON.28		CON.29		CON.30		CON.31		BHV.1		BHV.2		BHV.3		BHV.4		BHV.5
Variables	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE	Coef	SE
Intercept	1.67	1.6	4.01***	1.5	2.10	1.6	1.90	1.4	1.85	1.5	1.66	1.5	−0.28	1.7	1.10	1.4	1.79	1.4	0.78	1.6	−2.55	1.6	−2.89	1.8	−3.29**	1.7	0.91	1.2	0.62	1.2	−0.37	1.3	−0.40	1.3	1.21	1.3
WS	0.52*	0.3	0.26	0.3	0.42	0.3	0.25	0.3	0.18	0.3	0.22	0.3	0.35	0.3	0.75***	0.3	1.10***	0.3	−0.01	0.3	0.70**	0.3	0.24	0.3	0.99***	0.3	−0.13	0.2	0.56**	0.2	0.54**	0.2	0.30	0.2	0.33	0.2
WS1	0.76*	0.4	0.23	0.4	0.35	0.4	0.06	0.3	−0.10	0.4	−0.49	0.4	−0.06	0.4	0.83**	0.3	0.86**	0.3	0.68*	0.4	1.12***	0.4	1.47***	0.5	1.07***	0.4	−0.51*	0.3	0.05	0.3	0.39	0.3	0.18	0.3	0.11	0.3
WS2	−0.04	0.4	−0.45	0.4	−0.17	0.4	−0.05	0.3	−0.58*	0.3	−1.13***	0.4	−0.75**	0.4	0.20	0.3	−0.01	0.3	−0.48	0.4	−0.11	0.3	−0.51	0.3	−0.23	0.3	−0.91***	0.3	0.01	0.3	−0.47	0.3	−0.62**	0.3	−0.72**	0.3
Age	0.02	0.1	−0.09	0.1	−0.00	0.1	0.00	0.1	0.03	0.1	0.04	0.1	0.13*	0.1	0.03	0.1	0.02	0.1	0.07	0.1	0.20***	0.1	0.21***	0.1	0.23***	0.1	0.04	0.1	0.07	0.0	0.13**	0.1	0.11**	0.1	0.04	0.1
Gender	0.22	0.3	0.30	0.3	0.75**	0.4	0.09	0.3	−0.03	0.3	0.28	0.3	0.51	0.4	−0.32	0.3	−0.30	0.3	0.42	0.3	0.87**	0.3	1.01***	0.4	0.42	0.3	−0.25	0.3	0.60**	0.3	0.17	0.3	0.51*	0.3	0.18	0.3
Jabodetabek	0.03	0.3	0.23	0.3	0.03	0.3	0.07	0.3	0.55*	0.3	0.40	0.3	−0.08	0.3	0.21	0.3	0.39	0.3	0.64*	0.4	0.48	0.3	0.49	0.4	0.43	0.3	0.11	0.3	−0.48*	0.3	−0.54*	0.3	−0.17	0.3	−0.07	0.3
TBL	−0.33	0.3	−0.03	0.3	−0.34	0.3	0.18	0.3	0.11	0.3	−0.24	0.3	−0.31	0.3	0.11	0.3	−0.54*	0.3	−0.84**	0.4	−0.92***	0.3	−0.62*	0.4	−0.76**	0.3	0.21	0.3	−0.37	0.3	−0.07	0.3	−0.18	0.3	−0.38	0.3
SWSUS	−0.27	0.3	−0.54*	0.3	−0.29	0.3	0.17	0.3	−0.32	0.3	−0.21	0.3	0.11	0.3	0.66**	0.3	0.49*	0.3	0.27	0.3	0.31	0.3	0.48	0.3	0.51*	0.3	0.69***	0.3	−0.27	0.3	−0.02	0.3	−0.27	0.3	0.05	0.3
Threshold 1/2	1.88***	0.4	1.40***	0.3	1.55***	0.3	0.61***	0.2	2.19***	0.4	1.72***	0.3	1.99***	0.4	1.03***	0.3	1.31***	0.2	0.32	0.2	0.55**	0.3	0.64***	0.2	0.94***	0.3	1.62***	0.1	1.24***	0.2	0.30	0.2	0.82***	0.2	1.05***	0.2
Threshold 2/3	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	2.05***	0.2	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	3.27***	0.3	3.25***	0.3	2.13***	0.3	2.71***	0.3	2.46***	0.3	3.14***	0.3	2.43***	0.1	2.30***	0.1	1.88***	0.2	2.59***	0.2	2.75***	0.2
Threshold 3/4	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.	3.77***	0.3	3.78***	0.2	3.84***	0.2	n.a.	n.a.	n.a.	n.a.
Final LL	−53.9		−63.1		−53.3		−71.9		−60.5		−61		−50.8		−67		−71.9		−60		−61.1		−59.2		−62.7		−115.2		−112.6		−84.2		−86.8		−90.5
Initial LL	−58.6		−68.6		−58.1		−73.2		−64.4		−68.8		−57.2		−79.4		−86.6		−66.7		−75		−75.3		−79.9		−124.8		−121.9		−94.7		−95.7		−97
Chi-squared	9.3		10.9		9.4		2.6		7.8		15.5		12.9		24.8		29.4		13.5		27.9		32.2		34.5		19.1		18.5		20.9		17.9		12.9
Significance	0.311		0.205		0.302		0.952		0.451		0.049		0.112		0.001		0.000		0.095		0.000		0.000		0.000		0.014		0.017		0.007		0.021		0.114
Pseudo R2	0.079		0.079		0.081		0.018		0.060		0.112		0.113		0.156		0.169		0.101		0.185		0.214		0.215		0.076		0.076		0.110		0.093		0.066
N	91		89		87		88		88		88		87		88		90		90		90		90		89		92		92		92		92		91

Note 1: Coef: Coefficient; SE: Standard Error; WS: Participation of WS; WS1: Member of WS1; WS2: Member of WS2; TBL: Triple bottom line; SWSUS: Strong/weak Sustainability; LL: Log Likelihood; and n.a.: not available. Note 2: *: p < 0.1; **: p < 0.05; ***: p < 0.01.

.

With regards to the results related to CSI, Table 3 shows that the effect of participation in the workshop was significant in terms of CSI. 1, 2, 4, 6, 11, and 13. First, CSI. 1 (“We are not affected by negative environmental impacts that are caused by people’s activities in a city far away from our living place”) was negatively changed by the workshop because the workshop enabled the participants to recognize the potential negative environmental impacts from other cities. Second, CSI. 2 (“As income inequality gets larger, urban economic/social life becomes more unstable”) was positively affected by the workshop because the workshop promoted an understanding of how income inequity increases instability. Third, CSI. 4 (“Urban sprawl is the enemy of the global environment”) was positively influenced by the increased understanding of the negative impacts of urban sprawl. Fourth, CSI. 6 (“We should accept distinct urban lifestyles for accomplishing human sustainability”) was positively stimulated by the workshop because the workshop revealed how a diversity of urban lifestyles contributes to human sustainability. Fifth, CSI. 11 (“A city must be independent of other areas. A city does not need to rely on other areas”) was negatively shifted by the workshop, as participants became sensitized to the interdependence of cities on other regions. Finally, CSI. 13 (“Cities do not have large negative impact on human sustainability because urban areas account for only 0.2% of land surface on earth”) was negatively altered by the workshop, as participants realized that individual cities have significant negative impacts on human sustainability. Table 3 also indicates that age significantly affected the results for CSI. 4, 5, 10, and 13; gender for CSI. 1,2, 3, 4, 5, 7, 12, 16, and 17; Jabodetabek in CSI. 1, 4, and 7; Triple bottom line for CSI. 2, 3, 7, 9, 14, and 15; and, Strong/weak sustainability for CSI. 9 and 13, respectively. First, the results imply that older participants tend to have a more sustainable mindset than younger participants. Age positively influences CSI. 9 (“Lifestyles in developed countries are to blame for aggravation of human sustainability”), probably because more senior individuals tend to have more knowledge and/or experience about the history of economic prosperity in the developed world, leading them to believe that the developed world is responsible for the environmental issues. Second, the results suggest that the male participants tend to have a more sustainable mindset than female participants. The reason for this is unclear. Third, participants from Jabodetabek tend to identify themselves as main players in the local context.

In terms of the NEP results, Table 4 shows that the effect of participation in the workshop was statistically significant only for NEP. 7. This means that the workshop promoted the participants to think that flora and fauna is afforded the same right as humans. Table 4 also indicates that age significantly influenced the results for NEP. 3, 4, 5, 6, 10, and 13; gender for NEP. 6, 11, 13, 14, and 15; Jabodetabek origin for NEP. 3, 5, 6, 7, 11, and 13; Triple bottom line for NEP. 1 and 9; and, Strong/weak sustainability for NEP. 1, 5, 7, and 9. The results imply that older participants tend to believe that human beings have greater environmental impact and to consider environmental degradation more serious. This tendency is also found in male participants and those from Jabodetabek.

In terms of the results for CON (Questions regarding concerns about sustainability issues and responsible stakeholders), Table 5 show that the effect of participation in workshop significantly influenced CON. 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 19, 26, 27, 29, and 31. First, this indicates that participants became aware that they were one of the critical stakeholders. Following the workshop, the participants also expected the NGOs and countries to take greater responsibility for achieving sustainability. Furthermore, the participants did not regard absolute poverty and biodiversity as major issues although these are in reality serious issues. The results also indicate that age significantly influenced CON. 3, 13, 14, 18, 25, 29, 30, and 31; gender for CON. 5, 7, 8, 10, 18, 21, 29, and 30; Jabodetabek origin in CON. 2, 3, 5, 6, 12, 13, 23, and 28; Triple bottom line for CON. 13, 27, 28, 29, 30, and 31; and, Strong/weak sustainability in CON. 6, 13, 20, 26, 27, and 31. The results imply that older participants tend to have greater concerns regarding human sustainability.

Finally, for the results of BHV (Questions regarding intention of taking pro-sustainability or/and pro-environmental actions), Table 5 shows that the effect of participation in workshop significantly influenced BHV. 2 and 3. This indicates that the workshop motivated the participants to rely on politicians for overcoming income inequality and rely on environmental organizations for solving the environmental issues. The case story may have influenced the participants to consider that domestic political issues are the responsibility of politicians and that the environmental issues are the responsibility of international environmental organizations. Table 5 also indicates that age significantly influenced BHV. 3 and 4; gender for BHV. 2 and 4; Jabodetabek origin in BHV. 2 and 3; and, Strong/weak sustainability in BHV. 1. The results imply that older participants have greater intention to take action toward environmental protection; male participants tend to prefer solutions through the political process; and, the participants from Jabodetabek tend to distrust politicians in terms of income redistribution and to rely less on environmental organizations.

6.2. Discussion

First, our results revealed that more items in CSI (six items) were significantly affected by participation in the workshop than those in NEP (one item). This implies that the workshop stimulated the learners to achieve a more integrated view of sustainability in terms of multiple dimensions, which include the trade-off and/or integration of economy, society, and environment in the urban context. This may contribute to more holistic solutions. For instance, the learners could come to realize the trade-off relationship between the alleviation of negative environmental impacts and the reduction of consumption, which requires a more strategic economic approach and intervention.

Second, we obtained the findings indicating the growth of interests in environmental issues and the increase in a sense of responsibility. The participants learned that there were negative leakage effects of human activities in areas that were far away from their living places on their life (CSI. 1), understood that urban sprawl was the enemy of the global environment (CSI. 4), and recognized that cities were not independent of other areas and relied on them (CSI. 11). They may suggest that our education program enabled the participants to broaden their geographical scope and to gain a picture of interactions between their own city and other cities/regions in the world in terms of environmental issues, although it is often difficult to think of them in their usual life. The participants also increased their interests in environmental problems through the educational program while considering the consequences for the things of the future, such as their own future, children, and future generations, in addition to consequences for their life (CON. 5–CON. 13). They may imply their increased sense of responsibility for achieving sustainability in a future-oriented manner.

Third, the results also revealed that the workshop increased the participants’ awareness that they were one of the critical stakeholders. They enhanced their interests in environmental problems because of their impacts on themselves, their own life, children, people in the cities and the country in which they reside, their future, and future generations (CON. 5–CON. 13). This is because the case story gave more weight to people in the target areas and local issues, rather than global issues.

Fourth, the Case Method that was applied in our program stimulated the learners to understand the complexity of the sustainability concept and to comprehend the difficulties in consensus-building. One of the advantages of using the Case Method is that the protagonist is confronted with realistic human relationships and decision-making processes in terms of conflicting interests among stakeholders.

Fifth, based on our experience, a case story that relates to developing cities may enable learners to broaden their perspective on sustainability. Developing cities experience a diversity of socio-economic and socio-demographic patterns. The implementation of our teaching program in developing cities may encourage balanced viewpoints regarding sustainability. This type of education method could benefit developing countries.

Sixth, this education program that is based on city sustainability fosters five key competencies in sustainability that Wiek et al. [44] point out: systems-thinking, anticipatory, normative, strategic, and interpersonal competencies. Systems-thinking competence is required in city sustainability since cities depend on resources and ecosystem services from other areas. Our workshop requested the participants to take any linkages and leakage effects between local and global issues into consideration, and to also consider cyclical interactions that affect the triple bottom line on both local and global scales. They motivated them to gain the ability to analyze complex systems across different scales and across different domains. Actually, the results of our workshop revealed the enhancements of participants’ understanding of leakage effects (CSI. 1, CSI. 4, and CSI. 11) and cumulative global impacts (CSI. 13). Anticipatory competence is required to assess the multi-criteria assessment of the three future scenarios. The case involved a multi-criteria assessment for monocentric, spread, and polycentric scenarios in 2050, and it inspired the participants to become aware of the current problems that they were confronting in their own cities and the expected issues that their decisions could lead to in the future. They are reflected in the results that the participants’ concerns about their future (CON. 8), children (CON. 12), and future generation (CON. 13) were increased by the workshop. Normative competence is developed through multi-criteria assessments. The workshop requested the participants to compare multiple social values and examine their priorities through the group discussions with multiple members who may have diverse ways of thinking. In reality, our workshop strengthened the acceptability of diverse urban lifestyle (CSI. 6) and fears of social instability that are caused by increased income inequality (CSI. 2). Strategic competence is cultivated through the Case Method. The learners were required to arrive at their decisions from the viewpoint of a specific protagonist in the case story. The protagonist was the head of the urban planning bureau of DKI Jakarta. Thus, planning, decision-making, and organizational management should be considered in the context of social transition and transformation including stakeholders’ behavioral changes. They were found out in our workshop, through which the participants became more concerned about comprehensive issues relating to human sustainability in triple bottom line (CON. 26, CON. 27, CON. 29, and CON. 31) and they got to identify the stakeholders such as NGOs and national government, who should take further efforts to achieve sustainability (CON. 16 and CON. 19). Interpersonal competence is bolstered through the participatory processes and teamwork in the workshops. The case story enabled the participants to notice various stakeholders in the city, while the group discussion promoted them to understand multi-faceted aspects in each agenda and to examine the expected conflicts and trade-off among different stakeholders and perspectives. They could bolster the capacity to understand, embrace, and facilitate diversity across cultures, social groups, communities, and individuals [44]. This effect was observed in the fact that the participants got to respect the diversity in urban lifestyle for achieving sustainability (CSI. 6).

Seventh, our results indicated that gender and age had significant associations with sustainability-related indicators. The finding that male participants had higher interests/concerns regarding sustainability than female may be unexpected, because past studies generally suggested that women tend to have a higher sustainability mindset [62,63]. One of potential reasons is the self-selection bias that the male participants with more interests/concerns about sustainability might have voluntarily participated in the workshop. Meanwhile, the reason why senior students had higher interests/concerns about sustainability is firstly that more knowledge about sustainability through the education program in the university enhances the interests/concerns about sustainability in senior students; and secondly that the self-selection bias brings about the results that the senior students could have higher motivation of participating in the survey.

Lastly, our results show that the proposed education program using the case method has some limitations. One limitation is the bias toward urban-related issues because the case story focusses on urban issues. This could be overcome by extending the case story to consider non-urban aspects, including rural areas in the same country and/or other countries. Another limitation is the focus on specific environmental impacts that are related to the selected city, the Jakarta Metropolitan Area. Using an alternative city could lead to different educational outcomes. Thus, applying this method, the education program organizers should carefully select the target city to maximize the educational benefits of the program. In the same way, the selection of the protagonist in a case story may also affect the educational outcomes. Ideally, the protagonist should be selected, so that his/her decision-making directly influences multiple dimensions in the story. Additionally, this method requires considerable time and cost for producing teaching materials and for implementing workshops. The preparation of evidence of the impacts of specific actions on sustainability is challenging since it requires empirical analysis with scientific data and/or simulation models. Although the evidence could be hypothesized, it should be based on the realistic conditions in the given city. Finally, although we partly observed positive changes in their awareness of the behavior toward solutions to environmental issues and improvement in sustainability (BHV. 2 and BHV. 3), this does not mean that the participants changed their behaviors. It goes without saying that there is still a big gap between consciousness and behavior.

7. Conclusions

In this study, teaching materials were developed, educational workshops were designed to integrate city sustainability into sustainability education, and the educational outcomes of the materials and workshop were measured to identify the methods for enhancing the effectiveness of sustainability education.

The results were obtained by analyzing the responses of participants to pre- and post-workshop questionnaire surveys. They showed that a sustainability teaching program using the city sustainability concept is effective in higher education. First, the workshop incorporating city sustainability stimulated the learners to achieve a more integrated view of sustainability, higher interests in environmental issues, and a sense of responsibility in the urban context. This kind of education program in universities is more likely to be important for fostering young adults in urban areas who well understand sustainability and take leadership in guiding the society toward sustainable development, because more people are expected to reside in cities in coming years. Second, the adoption of Case Method with the city sustainability concept in the sustainability education also enabled the learners to understand the complexity of the sustainability concept and to comprehend the difficulties in consensus-building, particularly in a city. This approach motivated the learners to become aware that they were one of the critical stakeholders, and this may be one of the critical issues for people notably in developing countries, who may often assume that developed countries should take responsibilities of sustainability. Third, city sustainability contains the concept of triple bottom line. The sustainability education program with the concept of triple bottom line drove the learners to balance the multiple aspects of sustainability. This effect is important notably for people in developing countries, who tend to prioritize economic development over other sustainability elements. Additionally, our program highlighting city sustainability successfully fosters five key competencies in sustainability: systems-thinking, anticipatory, normative, strategic, and interpersonal competencies. This may suggest that sustainability education using the city sustainability concept could generally encourage people to have comprehensive capacities in sustainability.

Although the results of this study suggest that an education program for sustainability that incorporates the concept of ‘city sustainability’ changes concerns, attitudes, and motivation for action, the specific causal relationships among factors, such as attitude, sense of responsibility, and intention to act in a sustainability-oriented manner, were not clarified. Thus, further studies that investigate the psychological and social/structural mechanisms are necessary to better explain the effectiveness of incorporating the concept of city sustainability in sustainability education.