Assessing Sustainability of Wastewater Management Systems in a Multi-Scalar, Transdisciplinary Manner in Latin America

Abstract

Wastewater management in Latin America faces great challenges to reach a sustainable state. Although enough infrastructure has been built to treat around 40% of wastewater, only between 15–20% is effectively treated, and abandoned or defective infrastructure is a common sight. Data about current conditions at specific sites is quite fragmented, when existing. This leads to challenges in management, decision making and planning for sustainable options. We argue that a main obstacle is the lack of a regionally relevant sustainability assessment framework that allows for a holistic understanding of wastewater management as a nexus problem. We therefore developed a comprehensive framework to (1) understand current conditions (2) involve stakeholders and (3) point to pathways to improve wastewater management in the Americas. Building on literature review and stakeholder involvement, we constructed a multi-scalar extended dataset framework that is adaptable to different study sites using specific criteria. Sustainability was assessed through a “distance-to-target” approach. Social and economic variables were the lowest ranking in both cases, with technical variables generally performing better. Although some dimensions of sustainability are performing acceptably, others, such as social and economic, are general low to very low performing. This means, when looked at in an integrated manner, neither of the wastewater management systems analysed can be considered sustainable. Here we present the approach itself, the results of its application in two pilot sites in Latin America, and our recommendation to shift waste water management into sustainability.

1. Introduction

Wastewater and Its Management in Latin America

Wastewater management systems (WWMS) serve multiple functions within their cities. They channel and treat the wastewater produced by their customers, reduce the pollution load to the environment and the catchment they are embedded in and thus safeguard it and its inhabitants from detrimental health effects. Usually citizens only notice them when they do not provide those services. Wastewater treatment systems can, in addition, provide resources, such as bioenergy from biogas produced during the decomposition of organic matter, irrigation water or stabilized sludge to be used as fertilizer. Understanding the risks and benefits that a wastewater treatment system can offer to its community is not limited to the technical understanding of its components. It demands understanding the multiple dimensions of sustainability, understood as ‘the maintenance of economic well-being, protection of the environment and prudent use of natural resources, and equitable social progress which recognizes the just needs of all individuals, communities, and the environment’ [1].

In Latin America, 80% of the population lives in urban areas, with small cities (up to half a million inhabitants) growing the most rapidly [2]. Exact data on sanitation and treatment coverage are not readily available [3], but it is known that wastewater treatment is in general poor, with infrastructure to treat around 40% of municipal wastewater having been built, but less than 20% of that wastewater effectively being treated [4,5]. Commonly built solutions have been centralised wastewater treatment plants (WWTP), which may satisfy the demand of highly populated areas, but do not necessarily comply with the new expectations about water recycling and reuse, and of nutrient recovery [6], as requested by Sustainable Development Goal (SDG) 6.3 or the New Urban Agenda adopted at the latest Habitat III Conference [7].

Tackling the deficit of safely treated wastewater is an urgent matter: Clean water and access to safe sanitation for all is one of the targets decided by the global community within the Sustainable Development Goals (SDG 6.2) [8]. In Latin America large cities concentrate the largest shares of population, but when it comes to issues in the water management services, rural areas and small- and medium-sized cities are the most affected zones, especially regarding sanitation and wastewater treatment [4]. Small- and medium-sized cities are defined according to population, varying in proportion to each country’s size, with a maximum of 1 million inhabitants for Latin American cities [3]. These types of cities show high urbanization rates, being the fastest growing urban areas [9]. This means that the established urban management systems have to consider the growth projections and adapt to keep up with the growing water demand and wastewater generation. Therefore, sustainable options for wastewater management for small- to medium-sized cities are urgently needed.

The SludgeTec project, a multinational partnership (the United Nations University’s Institute for Integrated Management of Material Fluxes and of Resources—UNU FLORES, the Universidad de San Carlos de Guatemala—USAC, the Mexican Trust Fideicomiso de Infraestructura Ambiental de los Valles de Hidalgo in Tepeji, Mexico—FIAVHI, and the Technische Universität Dresden-TUD, aimed for international experts and local stakeholders to co-design a sustainable wastewater treatment and management options for two pilot areas in the Americas: Los Cebollales WWTP in Panajachel, Lake Atitlan, Guatemala and Tlaxinacalapan WWTP in Tepeji, State of Hidalgo, Mexico. Research was carried out between November 2017 and February 2019 by a multi-disciplinary and international team of researchers and practitioners.

To achieve the project’s objective (co-designing sustainable options), it was first necessary to accurately assess current sustainability, that is, to describe baseline conditions. Establishing baselines is crucial for scientifically sound sustainability interventions [10], and is a key practice in many environmental fields, as it allows to evaluate the change in time of given parameters and therefore to track project success, for example. Without a baseline, it is impossible to carry out “before and after” comparisons [11]. Furthermore, a baseline assessment can be very useful in informing and engaging stakeholders [9], and a powerful way to gather and centralize otherwise dispersed data, assess data availability for a given topic, and eventually, socialize knowledge. This is particularly relevant in a region where data scarcity is known to be an issue.

The importance of baseline setting being clear, we were confronted with the non-existence of a comprehensive guideline to describe baseline and assess the sustainability of WWMS. Guidelines exist on the broad and very general steps to be followed in establishing a baseline [12], and on the data items to be considered in the assessment of specific components of a WWMS, such as finance, technical issues, etc. [13,14]. There has also been some research to systematise the indicators and data items needed for technology options evaluation [15,16,17]. However, the guidelines analysed during our literature review focus mostly on single dimensions of sustainability (environmental, technical, social), and do not take into consideration broader scales of analysis beyond the WWTP itself (to include for example the impacts of the WWTP’s function on the watershed or the subcatchment). We posit that a sustainability assessment must be multi-scalar (considering several territorial scales or spatial boundaries in one same study) and multi-dimensional (considering the different dimensions of sustainability).

We therefore developed a method to describe baseline conditions of WWMS and determine the degree of sustainability by (1) constructing a comprehensive and adaptable dataset framework and (2) applying a “distance-to-target” approach (further described in the methods section).

The method is underpinned by an emphasis on participation and transdisciplinarity. Scientists in the field of Integrated Water Resources Management highlight that participation can have positive effects on finding integrated solutions, e.g., by gathering and exchanging knowledge between vital stakeholders [18,19]. In terms of specific WASH-related problems, participation can help identify acceptable solutions on the ground. Based on this knowledge, practitioners and especially international donor organisations, apply participatory approaches in various contexts [20,21].

A research approach in which scientific and non-scientific actors collaborate in a participatory manner with the aim of creating scientific knowledge meant to address practical problems is here understood as transdisciplinary research (e.g., Reference [22]). ‘Transdisciplinary’ generally refers to an intensive inclusion of practitioners in the research process. To conceptualize transdisciplinary research, research provides a set of design criteria that are likely to have an impact on addressing complex problems in practice. These design criteria refer to (i) the type of actors involved, (ii) the stage of the research process where these stakeholders are involved, (iii) the degree of their involved, and (iv) the respective methodology [23]. Hence, various actors have been involved at different stages of the research process, from the design of research projects, via the implementation of the research projects, up to the evaluation of research results. In doing so, research questions, methods, and results are possibly better adapted to local needs, accepted, and thus also implemented [22,24]. Transferred to the field of wastewater management, the involvement of different scientific disciplines and practitioners from different realms may enable an ecologically, economically, environmentally and socially sustainable treatment of wastewater.

Participation is however no panacea for successful solutions. To achieve the potential benefits of participation, the thoughtful design of participatory processes is essential, including the right mix of actors (e.g., households, farmers, public authorities), degrees of participation (e.g., information sharing or co-decision-making), at the right scale (e.g., local or basin scale) [22,25].

In brief, in order to codesign sustainable options for the WWMS at the pilot sites, we built a method to first assess baseline sustainability, considering different territorial scales and the environmental, technical, economic and social dimensions. To broaden the possibility of accurate understanding of the issue and successful outcomes of the project, we worked in a transdisciplinary manner, i.e., in a diverse scientific team which closely worked with stakeholders and local partners, in every stage of research.

2. Materials and Methods

The method consists of four ‘building blocks’: (1) A thorough understanding of baseline conditions, which are then assessed under three different but converging perspectives: (2) Sustainability Assessment (SA), (3) Stakeholder Analysis and (4) Wickedness Analysis (WA). Blocks 1 and 2 are consecutive, i.e., number one is needed to perform number two. Blocks 3 and 4 are carried out separately. The assessment is made more thorough and comprehensive by bringing in the specific knowledge of each building block. This facilitates the understanding of bottlenecks and pathways towards sustainability, and as a final outcome, makes it possible to envision and evaluate solution options (Figure 1).

Figure 1. The general method used in this research project. Highlighted blocks are the four building blocks in our method. This paper deals in detail with two blocks: Baseline Description and Sustainability Assessment.

This paper describes the first two building blocks in detail, while the remaining two are the object of future publications.

2.1. Pilot Sites

Pilot sites (Figure 2) were chosen by local project partners based on their knowledge of the reality on the ground.

Figure 2. Location of the pilot sites.

2.1.1. Panajachel Site description

At Panajachel, Guatemala, the pilot site is the Cebollales WWTP, an extended aeration, activated sludge plant built in 2013. The plant is operated by the municipality, with its financing sources being 100% public. The design flow is 37 liters per second (lps), and the current average flow is ~25 lps. It discharges into the San Francisco River, which, 200 m further downstream, feeds the Atitlan Lake. In the lake’s endorheic basin, 55% of households are connected to a sewage system, while the remaining 45% use latrines, septic tanks, or soak latrines. 45,500 m³ of wastewater is generated every day in the basin, and only approximately 20% receives treatment. Moreover, in the existing WWTPs, poor removal of pathogens and nutrients is a crucial challenge. These WWTPs face, among others, operation and maintenance problems.

2.1.2. Tepeji Site Description

At Tepeji, Mexico, the pilot site is the Tlaxinacalapan WWTP (built in 2017, started operations in January 2018). The WWTP has two treatment steps: a train of plastic anaerobic digestors built on site, followed by constructed wetlands. The design flow is 1.5 lps, and the current average flow is ~0.4 lps. It discharges into a tank from where water is taken to irrigate a football field and agricultural plots.

2.2. Dataset Framework

2.2.1. Preliminary Step. System Model: Boundaries and Scales of Analysis

Wastewater management is a wicked problem: a complex network of components, often interlinked in non-linear relationship and expanding across different territories. In addressing sustainability problems, systems approaches have been widely recognized to enable researchers to describe and understand reality more accurately, shedding light on a phenomenon’s structure and function [26,27,28], helping reveal otherwise “hidden” flows [29] and promoting the integrative thinking and interdisciplinary knowledge synthesis needed for sustainability [27,30]. System models are a key tool of systems approaches [27] and are widely used in cybernetics, physics, ecology and other fields where it is necessary to visually represent the complexity of real life networks and processes, in order to grasp the performance and behaviour patterns of systems. Our approach builds on systems thinking by using a system model as a fundamental research tool.

In building a system model, it is important to remember that, although WWMS are bound to human settlements, the sourcing of their inputs and the effluent and other outputs may have consequences well beyond their immediate geographical setting. Therefore, defining relevant scales of analysis and tracing analytical boundaries of the system is crucial. The choice of scales can determine the accuracy of diagnosis, and the effectiveness of projects [10,31]. Spatial resolution determines the visibility of objects and relations. If a model’s boundaries are too small, important factors influencing the model may be missed, whereas if they are too large, detail on specific processes may be lost. Avellán et al. [32] postulate that ‘the boundaries of the [Water-Soil-Waste Nexus] systems need to be (a) wide enough (to avoid microanalyses of plot levels as in some cases of INRM [Integrated Natural Resource Management]), (b) clear (to avoid confusion as in the WEF [Water-Energy-Food] Nexus), and (c) flexible enough to accommodate varying needs (to avoid geographic constrictions as is the case of the basin discussions in IWRM [Integrated Water Resources Management])’. By mixing in and contrasting different perspectives, a multi-scalar approach provides for more comprehensive analyses, which can lead to reduce biases caused by the use of a single “viewing-point” [10,33].

Different types of boundaries were identified: administrative (municipality, department, state, etc.), biophysical (catchments, geological, soil, etc.), and technical (treatment system, canal network, etc.). The relevance of each of these different spatial definitions was evaluated (Figure 3a), and four working scales were decided upon: 01 WWTP, 02 Municipality, 03 Subcatchment, 04 Watershed (Figure 3b). We argue that these scales together exhibit the needed specificity of the actual problem of wastewater treatment on the one hand but also enough scope to determine the impact that the system has on its surroundings.

Figure 3. (a) A first boundary explorations map for the Panajachel case, showing different scales of analysis that were initially found to be of interest: the plant scale (red dot), the subcatchment (light green), the municipality (yellow), the watershed (blue) and the province (orange). The large water body in the center is the Atitlan Lake. (b) The abstraction of “real-world” boundaries into boundaries for the modeling process. (c) The system model for the Panajachel case, showing the systems components in the scale they (mostly) operate in.

A system model for the WWMS was drafted for each study site, using these chosen boundaries. System components (stocks) are represented in boxes and relations between them (flows) with lines (Figure 3c). The first versions of the system model were refined with participation from stakeholders during an assessment workshop held in Panajachel, Guatemala, in March 2018 [34]. Figure 3c shows the final version of the system model for the Panajachel site, resulting from the participative work at the workshop.

2.2.2. Constructing the Dataset Framework

2.2.2.1. Extended Dataset Framework

We created the framework for a dataset that allows for a deep and holistic understanding of baseline conditions and sustainability performance, across scales and across the dimensions—environmental, and social, technical, economic—of the nexus problem of wastewater management, specifically in Latin America. To do so, we iterated between a top-down method (literature reviews) and a bottom-up method (working directly on the pilot sites, e.g., analysing the system model, asking stakeholders what sort of data is relevant to them) (Table 1). The result is an extended dataset framework (EF), which is described in more detail in the results section.

Table 1. Steps followed in the construction of the extended dataset framework.

	Bottom Up	Top Down
1	System model analysis	Research literature review
2	Stakeholder input (assessment workshop) on locally relevant data items and indicators	Policies and regulations review
3	-	Technical guidelines review

2.2.2.2. Site-Specific Dataset Framework

The EF was edited into a smaller, site-specific dataset framework for each site. This was necessary in order to respond to local data needs as expressed by stakeholders, as not all data items on the set were relevant for the specific sites. Additionally, to respond to the research priorities as established by the research team after assessing data availability and incorporating stakeholder input.

To edit the EF (with 492 variables) into the site-specific dataset frameworks (with 195 and 218 variables), we classified and prioritised each item on the EF according to the criteria in Table 2. Note that these criteria were chosen for these two project-specific pilot sites, but they could easily be applicable generically for other WWMS.

Table 2. Criteria used to prioritize the data items in the extended dataset framework and to create a site-specific dataset framework.

	Criteria	Priority
1a	Stakeholders chose the item during Assessment Workshop	P1
	PLUS (+)
1b	Literature on wastewater management mentions it
2	Locally applicable regulation calls for the parameter	P2
3	Thresholds to compare current value against are available	P3

Priority 1 (P1) was given to an item if two conditions are met: (a) that stakeholders had chosen it during the Assessment Workshop held in March 2018 in Panajachel, and (b) that the item had been found in wastewater management guidelines or other relevant literature during our literature review. Priority 2 (P2) was given when the data item is included in relevant local regulation, e.g., monitoring standards. Priority 3 (P3) was given when a threshold to compare current values to could be identified. Thresholds were found looking in:

• Local legislation (region, state, basin).
• National legislation.
• Legislation valid for the other case study of this project (in this case Mexico or Guatemala).
• International organisations (not legally binding but accepted as guidelines or recommendations).

In some cases, a data item is a “yes or no” question, and a threshold can be established with relative ease; for example, the existence of an operation manual for the plant, for which the threshold is “yes”, since that would be the desirable situation.

After the data gathering phase (see Section 2.4), these dataset frameworks were “filled in” with data, allowing to understand baseline conditions and perform a sustainability assessment.

2.3. Data Gathering

2.3.1. Identifying Data Holders

Possible data sources were identified through (1) an Assessment Workshop and (2) deskwork. The Assessment Workshop took place in March 2018 in Panajachel, Guatemala. Stakeholders from the Mexican cases were also present. More than 80 local stakeholders were invited, of which a total of 39 participated. The represented stakeholder groups were coming from Academia (43%), Federal officials (21%), Non-Governmental Oirganisations (NGOs) (20%), Municipal officials (8%), and Private enterprise (8%). Through the participatory activities crucial input needed to refine both the technical and the social assessment components of the project framework was obtained. A thorough comprehension of the current problem structure, made possible by including the views of key stakeholders in a very interactive and participatory manner [34]. Participants drafted lists with institutions and experts who they thought could have the information needed (or access to it) for each data item. The data holder lists made by the participants were screened, refined and complemented through desk work. The final list of data holders consulted or interviewed can be seen in Appendix C.

2.3.2. Data Collection

Part of the data needed was collected on the field. Fieldwork was carried out for two weeks at each site in August 2018, and included meetings with experts, practitioners and local authorities identified as data holders, as well as sampling and laboratory analysis. During the meetings, the interviewer (L. Benavides) went through the dataset with the stakeholder, who provided the answers he or she had available. Data holders were always asked to provide supporting documentation, but this was rarely available. In some cases, stakeholders did not have information at hand, but committed to sending it via email after the meeting.

For water quality parameters, sampling and laboratory analysis were carried out (Appendix D). Sampling and analysis were done in accordance to the norms in each country, in collaboration with certified local laboratories. In both cases, a composite sample of both the plant’s inflow and outflow was taken during a 24-h period. At the Panajachel site, sludge was also sampled. The sludge available had been stabilized on a covered drying yard for 28 days and was then piled up outdoors (i.e., under sun and rain) for at least two months prior to our visit. In Tepeji it was not possible to sample sludge, as according to the managers, the plant had not produced any in the 8 months of operation.

Further data was obtained from the revision of literature and documents produced by local and national authorities, which were made available to the research team during field work.

2.4. Sustainability Assessment

Sustainability Assessment (SA) processes aim at guiding decision-making towards sustainability [35] using different evaluative techniques [36] and definitions. ‘Sustainability Assessment’ can be considered a broad name tag for a series of methods and approaches: e.g., Sustainability Appraisal, Integrated Assessment, Integrated Sustainability Assessment, Sustainability Impact Assessment, Triple Bottom-Line Assessment, 3-E Integrated Assessment and Extended Integrated Assessment [35,36,37]. Methodologies found for SA include multicriteria approaches, systems analysis, life cycle analysis, economic analysis (cost-benefit analysis, life-cycle costing, etc.), weighting methods (exergy analysis, entropic weighing method), distance-to-target approaches, among others [15,38].

To determine the level of sustainability, we used a “distance-to-target” approach, comparing the current value of a variable with the threshold previously identified (see Section 2.2.2.2). The availability of a threshold finally defines whether a data item could be used in the sustainability assessment or not. Even though data for an item is available, if there is no appropriate threshold to compare it with, it is impossible to profit from this already existing data. Appendix E lists the variables for which thresholds could be identified and the thresholds values used to evaluate each variable of the site-specific dataset framework.

The “distance-to-target” was evaluated by adopting the “traffic light” method [39], where a variable is coded with green if it meets the threshold (good performance), with yellow when its performance does not meet ideal standards but is not far away from doing so, and red when it is performing sub-optimally. Table 3 discloses the quantitative criteria for each colour. Each variable was evaluated following these criteria. The result is a colour-coding of the data set (Appendix F).

Table 3. Colour ranking in Sustainability Assessment.

Data Type	Criteria	Ranking
		Red	Yellow	Green
Real number	10% tolerance	MV > TH × 1.1	TH < MV ≤ TH × 1.1	MV ≤ TH
Percentage	Range divided into 3 equal parts (33% each)	MV < 33% or 67% ≤ MV	33% ≤ MV < 67%	67% ≤ MV or MV < 33%
Absolute values (e.g., yes/no questions)	No yellow range, unless mentioned otherwise	YES/NO Present/Absent Outside pH range	-	YES/NO Present/Absent Within pH range
Social variables (dataset IIb)	Scale 1 to 4	1 ≤ MV < 2	2 ≤ MV < 3	3 ≤ MV ≤ 4

MV: measured value; TH: threshold (numeric thresholds where normally defining a maximum, not a minimum).

Once the colour ranking was calculated for each variable, a colour ranking was also calculated for each of the three dimensions into which the variables are grouped in the sets: technical-environmental, economic, and social. To do this, we also followed the method described in Bertanza et al. (2016) [39], where a numeric value is assigned to each colour:

• Green = 1
• Yellow = 0
• Red = −1
• The colour-values are added, and a simple average in each category is calculated.
• The results are later presented again using the “traffic light” colour-coding for the performance of each dimension of sustainability, as follows: (see results section)

• Green: >0.33
• Yellow: between −0.33 and 0.33
• Red: ≤−0.33.

Although as stated in the introduction we believe a multi-scalar approach is necessary for a wide-enough perspective and an accurate understanding of a WWMS, due to the limited time scope of the SludgeTec project and the prolonged waiting periods to obtain data from data holders, it was only possible to perform a sustainability assessment on the first scale (WWTP, grey shaded areas on, and to include a multi-scalar social assessment of participation and social acceptance in the region where the WWTP operates (dataset IIb on Table 4 and Table 5).

Table 4. Extended dataset framework, overview of subsets and number of data items in each.

Subset	Description		Scales		Number of Data Items
Dataset 0 Context indicators	Understanding of context: geographical location and characteristics, poverty and employment indicators, etc.	50 data items for 4 scales	01	WWTP	7
			02	Municipal	18
			03	Subcatchment	13
			04	Watershed	12
Dataset I Technical-Environmental	Technical and environmental variables (e.g., population served, chemical parameters of water bodies and of effluents, WWTP management)	380 data items across 4 scales	01	WWTP	211
			02	Municipal	31
			03	Subcatchment	70
			04	Watershed	68
Dataset II Socio-Economical	Economic, financial, budget variables. Dataset IIb useful to understand the social acceptance of the system	IIa. 52 data items for 4 scales	01	WWTP	16
			02	Municipal	17
			03	Subcatchment	7
			04	Watershed	12
		IIb. 10 data items, across scales		Social space (cross-scale)	10
Total data items					492

Table 5. Site-specific dataset frameworks for both pilot sites, after prioritizing the EF.

Tepeji Dataset Framework			Panajachel Dataset Framework
Dataset	Scale	Number of Items	Dataset	Scale	Number of Items
Dataset 0 Context	01	3	Dataset 0 Context	01	1
	02	3		02	0
	03	4		03	0
	04	5		04	0
	Total	15		Total	1
Dataset I Technical Environmental	01	107	Dataset I Technical Environmental	01	98
	02	15		02	15
	03	15		03	55
	04	18		04	18
	Total	155		Total	186
Dataset IIa Social-Economic	01	7	Dataset IIa Social-Economic	01	8
	02	5		02	8
	03	0		03	0
	04	3		04	5
	Total	15		Total	20
Data IIb Multi-scalar Social	Total	10	Data IIb Multi-scalar Social	Total	10
Total items in framework		195	Total items in framework		218
Grey shaded areas indicate the data that used in sustainability assessment

3. Results

3.1. Dataset Framework

3.1.1. Extended Dataset Framework

The iterative collection process of data items to describe the multi-scalar WWMS resulted in a large dataset framework with 492 data items (for an overview see Table 4, for the full content see SM 1). This comprehensive or “extended” dataset framework contains data items useful for the transdisciplinary study of WWMS (environmental and technical, economic, and social factors). It is organised into three datasets, namely: Dataset 0 which describes generic context data, Dataset I containing technical and environmental data, and Dataset II, containing socio-economic data. All datasets contain information across the four different spatial scales identified in Section 2.3.1. (WWTP, municipality, subcatchment, watershed) (see Table 4).

3.1.2. Site-Specific Dataset Framework

The EF proved too extensive to be used for the assessment of the sites, as time was a limiting factor, and also because not all variables on the set were necessarily a priority or the data for needed for all was not available at the different sites. Therefore, from the 492 data items in the EF, a site-specific dataset framework was created for the Panajachel pilot site with 218 data items, and for the Tepeji pilot site with 195 data items (Table 5). The full site-specific dataset frameworks can be found in Appendix A for Panajachel and in Appendix B for Tepeji.

3.2. Data Gathering

Figure 4a,b show the distribution of sources from which the data came from.

Figure 4. Data sources per type. (a) Panajachel pilot site; (b) Tepeji pilot site.

About 77% of all data items in Scale 01 could be gathered for the Panajachel site, and ~76% for Tepeji (Table 6 and Table 7). However, out of the data that was gathered, only a fraction was of use, as can be seen in the last column of Table 6 and Table 7. The reasons why some of the data had to be discarded were:

Table 6. Data gathered for all scales, data gathered specifically for Scale 01 and data finally computed into the sustainability assessment: Panajachel pilot site.

All Scales: Data Found				Scale 01: Data Found			Scale 01: Data Found and Useful
	Total Items	Items Found	% Found	Total Items	Items Found	%	Number of Items Found and Useful	%
Dataset 0	1	1	100.00	1	1	100	- *	- *
Dataset I	186	88	47.31	98	73	74.49	52	71.23
Dataset II	31	23	74.19	18	16	88.89	10	62.50
Total	218	112	51.38	117	90	76.92	62	68.89

* NOTE: Dataset 0 contains context data and was not used directly in the sustainability assessment.

Table 7. Data gathered for all scales, data gathered specifically for Scale 01 and data finally computed into the sustainability assessment: Tepeji pilot site.

All Scales: Data Found				Scale 01: Data Found			Scale 01: Data Found and Useful
	Total Items	Items Found	% Found	Total Items	Items Found	%	Number of Items Found and Useful	%
Dataset 0	15	10	66.67	3	3	100.00	- *	- *
Dataset I	155	93	60.00	107	81	75.70	48	59.26
Dataset II	25	18	72.00	17	12	70.59	7	58.33
Total	195	121	62.05	127	96	75.59	55	57.29

* NOTE: Dataset 0 contains context data and was not used directly in the sustainability assessment.

• Data quality. Stakeholders sometimes provided no supporting facts or documentation for the data they provided, or there was a considerable difference between data found for the same item from various sources, with no straight-forward way to choose amongst them.
• No existing threshold. The data could be obtained but no threshold was found, and therefore the data was not used further.

This filtering process removed ~32% of the data for Panajachel and ~43% of the data gathered for Tepeji. With the remaining variables (62 variables for Panajachel, 55 variables for Tepeji) the sustainability assessment was performed.

3.3. Sustainability Assessment

3.3.1. Panajachel

Sustainability at the Cebollales WWTP in Panajachel was assessed with 62 variables: 52 in the technical-environmental dimension, three in the economic and seven in the social. All dimensions show a medium performance (yellow) except for the economic dimension, where the assessment is “poor” (coded with red). Just about half of the variables are performing relatively well (23 variables coded with green) and about half are coded in red (27), with two variables coded in yellow. Therefore, overall sustainability performance can be classified as medium to low (See Table 8. To see performance per variable, see Appendix F).

Table 8. Sustainability performance per dimension: Panajachel site.

	Variables Per Category				% Variables Per Category				Dimension Average
Dimension	R *	Y	G	Total	R	Y	G	Total	Value	Colour
Technical-Environmental (TE)	27	2	23	52	52%	4%	44%	100%	−0.08	Y
Economic (Ec)	3	0	0	3	100%	0%	0%	100%	−1.00	R
Social (S)	2	1	4	7	29%	14%	57%	100%	0.29	Y
Total or Average	32	3	27	62	60%	6%	34%	100%	−0.26	Y

* R = Red. Y = Yellow. G = Green. ND = No Data.

In the technical environmental dimension variables that performed well included heavy metal concentrations in the plant’s water outflow and sludge which were all found to comply with norms at the moment of sampling (except for Arsenic in the sludge). Additionally, the Sampling frequency is complied with. The plant is sampled with regularity. However, the results do not make it to on-the-ground stakeholders (plant operator, for example).

In contrast, variables performing sub-optimally include nutrients and organics: variables such as Total Nitrogen (TN), Total Phosphorus (TP), Biological Oxygen Demand (BOD), Chemical Oxygen Demand (COD), and Total Coliforms. The plant is not able to meet treated water outflow standards. This situation is likely partially driven by the fact that the inflow to the plant (municipal sewage) is already non-compliant as per regulations on discharges into the public sewage system. In addition, strong odours were detected while visiting and were also reported by local stakeholders. Lastly, maintenance is very irregular to non-existing; salaries are irregularly paid; no operation manual exists on site, the operators lack training and equipment. The risks that the WWTP and the treated water discharge into the nearby San Francisco river poses to health or to environment are unknown, as no risk assessment has been carried out, either for health or for ecosystems.

In the economic dimension there is no compliance. For example, the per capita cost of treatment is higher than the WHO illustrative value for activated sludge plants (upper limit set by WHO is 8 USD per capita per year. Using data provided by the municipality, we calculated 9.7 USD). The budget deficit is constant, i.e., the operating entity practically never has access to enough resources to cover operating costs or deliver worker’s salaries on time. There is also no valorisation of by products (biogas, sludge), nor has a plan for this purpose been outlined by managers.

In the social dimension stakeholders are generally aware and interested in wastewater-related issues and see opportunities for their suggestions to be heard. They however do no not perceive the solution(s) currently in place as acceptable, nor do they perceive that others accept them.

3.3.2. Tepeji

Sustainability at the Tlaxinacalpan WWTP in Tepeji was assessed with 55 variables: 48 in the technical-environmental dimension and 7 in the social dimension. No economic data was available from the WWTP managers at the time of data gathering, and therefore this dimension could not be evaluated. In the two dimensions evaluated, it shows a moderate to good performance (Table 9).

Table 9. Sustainability performance per dimension: Tepeji site.

	Variables Per Category				% Variables Per Category				Dimension Average
Dimension	R *	Y	G	Total	R	Y	G	Total	Value	Colour
Technical-Environmental (TE)	15	0	33	48	31%	0%	69%	100%	0.38	G
Economic (Ec)	0	0	0	0	ND	ND	ND	0%	ND	ND
Social (S)	2	2	3	7	29%	29%	43%	100%	0.14	Y
Total or Average	17	2	36	55	ND	ND	ND	ND	ND	ND

* R = Red. Y = Yellow. G = Green. ND = No Data.

The technical-environmental dimension performance’ falls just above the border between medium and good performance, with 33 out of 48 being coded with green. Variables that perform well include compliance with heavy metal concentrations in the outflow (as established in local regulations, i.e., Norma Official Mexicana (NOM) 001), except for Cadmium (0.02 mg/L, which is double the allowed value). Additionally, all physical parameters are complied with (Total Suspended Solids (TSS), conductivity, colour, floating matter, grease and oils).

Variables that perform sub-optimally include nutrients and organics: TN, Faecal coliforms, pH, are not performing satisfactorily, neither when compared with the local norm (NOM 001) or with WHO standards for the use of treated wastewater use in agriculture. Odours were detected while visiting and were also reported by local stakeholders. No operation manual is available to key stakeholders such as the operator himself. No regular sampling seems to be occurring on the plant’s outflow, as although some interviewed stakeholders assured sampling has been done, no results were provided to us. Operators and managers lack adequate training on anaerobic plant operation. Standard design and operation practices are not being followed (such as an initial inoculation of the system with appropriate bacteria at the start of operations, assurance condition of air-tight conditions within anaerobic digestion tanks). Finally, the risks that the WWTP poses to health or to environment have not been studied, either prior to construction or once in operation, by any of the possibly interested parties.

In the social dimension stakeholders indicated that they are interested in and aware of wastewater related problems. They however do not feel that there is enough information available or opportunities to participate in decision making or to give recommendations to decision makers and managers. The current wastewater management system is generally not accepted or perceived as being accepted by interviewed stakeholders.

4. Discussion

4.1. Dataset Framework for Describing Wastewater Management Systems

We designed a transdisciplinary approach to assess baseline conditions and sustainability performance of wastewater management systems in Latin America, building on methods from both the social and the natural sciences (Figure 1), and with a heavy emphasis on stakeholder involvement and the understanding of baseline conditions. The approach was designed along with the development of a research project, in an iterative process between academic knowledge and the real experiences of what was possible to achieve within the conditions on the field.

We created an Extended dataset framework (EF, see Section 3.1), which we propose to be useful as a general guidance for data item selection for WWMS. It can be used as a sort of repertoire that can be “curated” or edited, choosing the items that are relevant to a specific site or research question, and thus creating a site-specific data framework.

4.1.1. Methodological Issues

The approach calls for not only a transdisciplinary but also a multi-scalar assessment. We attempted to simultaneously look at local scales within technical and administrative boundaries (the WWTP and the municipality), and ecological scales within hydrological boundaries (subcatchment and watershed). However, gathering, evaluating and processing the data required for a multi-scalar assessment was impracticable within our time scope. We therefore implemented the approach only on the scale of the WWTP (Scale 01) and were able to gather enough data to describe baseline conditions and to assess sustainability across all dimensions of sustainability in Panajachel and two out of three in Tepeji.

The approach proved to be practicable at one scale, with the strength of being able to incorporate local needs and conditions through the site-specific editing of the extended dataset framework (Table S1, see Section 2.2.2.2). The resulting datasets are useful as snapshots of the current status quo, and the data items can be used as a guideline for future data generation and periodical evaluation.

Building on a systems perspective, this approach calls for the construction of a system model as a tool: to identify important data items or variables to be investigated, or to localise “invisible” parts of the system, such as stakeholders, boundaries or legal frameworks. The tool proved useful not only for our own research process, but was also helpful during stakeholder involvement activities, where it helped structure the discussion. One example is that when discussing key stakeholders and responsibilities (“who is involved and who is responsible for what?”), the system model clearly depicts that, because the WWTP’s outflow eventually reaches the lake at the bottom of the basin, basin authorities (federal), river authorities (provincial) and tourists visiting the lake are involved, albeit to different degrees, in the problem. In other words, by explicitly linking upstream sewage system users to downstream fishermen affected, for example, the visual representations appeal strongly to very different stakeholders sitting around a discussion table, promoting a holistic and inclusive understanding of issues.

The same can be said for the boundaries discussion, also guided by the system model. By clearly illustrating which components fall into which boundary (e.g., the whole of the WWTP falls within the municipal boundary, but its outflow, ten meters ahead, falls into the river and thus provincial jurisdiction, while some of its inputs, such as pump parts, come from a different continent) administrative responsibilities can be made clearer and better understood; conflicting interests or overlapping mandates are made visually explicit and can therefore be more easily comprehended, and complexity is more easily grasped.

Finally, the combination of a technical-environmental assessment (Dataset I) with an economic (Dataset IIa) and a social (Dataset IIb) assessment proved not only enriching but allowed for insight into the drivers of the technical and environmental results. The technical-environmental variables provide an answer to the question “How is the system behaving?” while the social and economic data provide perspective into “Why the system is behaving so?” making the method better poised to identify bottlenecks and point to solution pathways. We see a challenge but a promising opportunity to improve sustainability thought and its tools in a more thorough transdisciplinary integration in the future. Overall, the approach showed potential for investigating the sustainability of WWMS. We see areas of improvement in, for example, reducing data intensity, systematising thresholds, and operationalizing the multi-scalar approach.

4.1.2. Data Availability

As shown in Table 6 and Table 7, roughly over 50% of the data we originally set out to gather for the multi-scalar site-specific dataset framework(s) was available with certain ease of access. Once we decided to focus on a single scale (WWTP), this proportion grew to ~75%, of which around a third had to be discarded due to quality issues.

Basic information such as monthly or yearly budgets and expenditures records, technical drawings and plans of the WWTP were for example not available for the Mexican case. In the Guatemalan site, non-continuous time series for monthly expenditures, inflow and outflow measurements were finally obtained via email after a waiting period following a stakeholder interview. Although indeed useful, the time series were neither long, nor gap-free, and data was not easily made available. In general, we found that stakeholders who, in theory, should have information (operating facilities, government bodies, WWTP managers) may be able to provide verbal answers in an interview, because of their empirical knowledge. The however very often lack supporting documentation, written records and systematic registries. In other cases, they lack the willingness or permission to share information. This is true mostly at the municipal and state levels, while federal agencies, particularly in the Mexican case, usually have well integrated and functional databases. The scale of federal-level data is however often not fine-grained or detailed enough to study a single treatment plant or even a municipal-level WWMS.

It is clear that large efforts are needed in terms of data generation, systematisation, sharing, and transparency. Examples would be digitising written records, using same standards throughout the region, information sharing between institutions, researchers and stakeholders or placing documents and data on the internet. Good starting points already exist, such as the National System for Water Information, kept by the National Water Commission in Mexico (CONAGUA), where geo-referenced information on water quality and quantity, irrigation, and watersheds is disclosed. We suggest that an immediate area of work should be furthering the capacity of key stakeholders, such as municipal and state or provincial governments to generate data, and the integration of all data generators into more detailed and/or numerous data bases, or conversely the creation of citizen-led observatories that foster awareness raising and demand and contribute to regular environmental and economic monitoring.

A significant issue to meaningfully assess sustainability stems from the still incipient integration of social indicators into sustainability. In the extended dataset framework 10 indicators (versus 380 for the environmental-technical dimension) across all scales could be identified from the literature or the stakeholder discussions. They are often linked to information that is not readily available but has to be generated via questionnaires and on-site interviews, coded analysis and other qualitative research methods. In order to strengthen future sustainability assessments of WWMS it is imperative to continue work on the integration of social indicators and methods to streamline the collection and analysis of these.

4.2. Sustainability of the Pilot Wastewater Management Systems

Our analyses show that the wastewater treatment systems currently in place in Panajachel, Guatemala, and Tepeji, Mexico, although performing well in various selected parameters, cannot be considered sustainable when looked at in a multidimensional manner, i.e., in terms of technical, environmental, economic, and social factors.

4.2.1. Technical-Environmental Issues

Both plants treat a municipal wastewater flow of domestic origin, with low to negligible heavy metal and metalloids content. In both cases, the quality of the inflowing wastewater is already below locally applicable standards for discharges into the public sewage network (Total Nitrogen and Total Coliforms in both cases, and Total Phosphorus, Biological Oxygen Demand, Chemical Oxygen Demand, and Total Suspended Solids (TSS) in Guatemala as well) meaning the plants are receiving a low-quality inflow from the start (see results per variable in Appendix F).

Once within the WWTPs, processes show different efficiency levels. Omitting metals, in which both plants practically fully comply (arguably because of an original low-metal content), the WWTP in Panajachel does not comply with virtually any of the examined physical, chemical or biological parameters, while the Tepeji WWTP performs slightly better, complying with half of them. Both plants, however, are performing poorly in the treatment of faecal coliforms, a crucial variable in terms of human and ecosystem health. In the particular case of Tepeji, where the water is being used for irrigation and the de-centralised, small-scale technology is being introduced to the community, a low quality and potentially risky outflow is not only a health risk, but an important hinderance to the success of the de-centralised treatment project, which has the aim of fostering wastewater use for agricultural irrigation. Social acceptance is key to the success of such new technology, and thus trust among the community has to be gained by the promoting entities.

Technical efficiency and environmental compliance are a major issue in both plants. Although visibly more critical in the Mexican case, lack of training of the operating and management personnel is a shared issue that is contributing to the situation. Systematic data generation and environmental monitoring, particularly in the Mexican case, are a challenge. Guatemalan managers were keeping a more detailed track of the WWTP performance. This may have to do with the fact that the WWTP in Guatemala has been in operation for a longer period (5 years versus 8 months), and also that it is operated by the municipality (vs a Trust).

4.2.2. Management Issues

In management-related issues both WWTPs perform the same, with only 2 out 7 management variables evaluated as positive, even though the systems operate at different scales (design flows of 1.5 lps in Tepeji vs 37 lps in Panajachel on average). Both plants lack operation manuals accessible to the operator, personnel lack training and capacities and laboratory analyses are not accessible and hassle-free in any of the cases (although in Panajachel sampling is carried out with norm-compliant frequency by a federal authority, key stakeholders—such as the plant’s operator—do not have access to the results and have therefore no feedback on their work).

In terms of risk and safety, operators in both pilot sites lack appropriate working conditions, (clothing, equipment, adequate-hand-washing-facilities). In neither of the cases had the risks posed to the environment or surrounding populations by malfunctioning of the WWTP been studied. Panajachel stakeholders manifested that environmental risk assessment is relatively new in public administration, and that they hope it will be integrated along with, for example, Environmental Impact Assessment (EIA), soon.

In Latin America in general, investments are often made to build an infrastructure project, but the funding for its long-term operation and maintenance (including equipping operators, performing routine samplings, etc.) is not secured, and nor are income generating options (resource recovery, for instance) duly considered [40]. Although this is a known issue, new infrastructure is being built as we write in Panajachel, while in Tepeji funds are being sought for the building of a large scale WWTP, still without a clear idea of how current infrastructure will continue to be financed or maintenance challenges faced (e.g., equipment repairs, salaries). Without a change towards adequate financial planning it is likely that both existing and new WWTPs at the studied sites will continue to operate sub-optimally.

4.2.3. Social Issues

The overarching recommendation, applicable to both sites, is to facilitate stakeholders the access to the information about their own social network. A common understanding of the problem itself is lacking. Who should be contacted with which need, or as formulated by Reed et al. “who is in and why?” [23] is a key question with a high degree of influence on the social development in both pilot sites. A common understanding of the problem is the basis for facilitating social interaction among the involved stakeholders. Economic and human resources should be provided to conduct an in-depth Stakeholder and Social Network Analysis in both pilot sites.

5. Conclusions

To advance towards sustainability in the urgent topic of wastewater management in the Americas, data scarcity and scatteredness must be overcome to allow for precise understanding of current or baseline sustainability performance. From such an understanding, bottlenecks can be made visible, and pathways towards sustainability can be envisioned. To increase the accuracy of the assessment and the adequacy of proposed solutions, research should go beyond one single perspective. To this end, we have proposed a multi-scalar data framework that includes variables for four different territorial scales: the WWTP, the municipality, the subcatchment and the watershed. Other scales could be chosen in other projects, what we propose is the multiscalar approach, not necessarily the scales themselves. Additionally, we propose to assess sustainability across four dimensions (environmental, technical, economic and social), and to incorporate other strands of scientific practice into the assessment (stakeholder analysis and wickedness analysis).

Transdisciplinarity is also a tool for improved success of research projects in this topic (see introduction). Throughout this project, we worked closely with local stakeholders and non-scientific practitioners. Their input was crucial in tailoring the framework to be locally relevant (see Section 2.2.2.2), and in the process of envisioning and evaluating solution options.

In this paper we present the method itself (Section 2) and partial results of its application in two pilot sites (Section 3). We also discuss the benefits and limitations of the method, and point to ideas for its future improvement and further application (Section 4.1).

As to the method itself, we found the multiscalar approach to enrich assessment and to allow to make visible issues that are not shown by single scale analysis, namely the interconnections of the technical system (WWTP) with ecological systems (watershed, riparian areas) and social systems (government, public administration, community dynamics, social perception). Shedding light on these interconnections, bottlenecks and obstacles to achieve sustainability are understood in a deeper and more detailed way, as many of the bottlenecks would be invisible when looking only at one scale or one dimension. The main limitations of the method are data and time intensity. Good planning, working closely with engaged local partners and performing a preliminary screening of data availability and data holders is recommended.

As to the results of the assessment presented here, Sustainability Assessment showed that technical and environmental variables tend in general to perform medianly to well, with microbiological parameters performing below the norms in both cases. Social and economic variables are the weakest spot of both of the WWMS analysed (Section 3.3). The results of the other two components of the method (stakeholder analysis and wickedness analysis) will be the object of future publications.