Quality of Urban Water Services Provided to Users: Assessment System and the Portuguese Path through Four Generations, Lessons Learned and New Challenges

Abstract

Public water supply, urban wastewater and stormwater management, and urban waste management are structural public services, essential to well-being, public health, and the safety of populations as well as to economic activities and environmental protection. These services must be guided by principles of universal access, continuity and quality of service, and efficiency and fairness of applied tariffs. The main concern of the regulation of these services is the protection of the interests of the users through the promotion of the quality of the service provided by the water utilities and the guarantee of balance in the practiced tariffs, materialized in the principles of universality, equity, reliability, and cost-efficiency. The quality of the urban water services has been assessed by ERSAR (the Portuguese regulator) since 2004, when the first generation of the assessment system was developed, and has undergone three periodical critical revisions. The fourth generation, developed in 2021, entered in force in 2022. This paper presents the fourth generation of ERSAR’s system for assessing the quality of urban water services in Continental Portugal, focusing on the path followed and addressing the experience of its application over almost two decades, the lessons learned, and the new challenges for the water sector.

1. Introduction

Activities of public water supply to populations, urban wastewater management, and urban waste management are structural public services, essential to the well-being, public health, and safety of populations as well as to economic activities and to the protection of the environment. These services must be guided by principles of universal access, continuity, efficiency and quality of service, and fairness of applied tariffs.

Therefore, the main concern of the regulation of these services is the protection of the interests of the users through the promotion of the quality of the service provided by the water utilities (WUs) and the guarantee of balance in the tariffs practiced. These materialize the principles of universality, equity, reliability, and cost-efficiency [1]. Conditions of equality and transparency must also be guaranteed in the WUs’ access to the activity and in the respective exercise as well as in contractual relations, when applicable; safeguarding the economic–financial, infrastructural; and the operational sustainability of the systems. These conditions apply regardless of the public or private, municipal, or multi-municipal status of the WU. Safeguarding the rest of the sector’s business, which is not regulated but supports the WUs and the environmental objectives, must also be ensured.

The Portuguese Water and Waste Services Regulation Authority (ERSAR) pursues its mission of regulation and supervision of water services. Its regulation model aims at contributing to ensuring:

• The overall sustainability of the sector, through an adequate national strategy, a sound legal framework, the existence of information, and a permanent innovation effort (supported with research and development results);
• Social sustainability, through physical and economic accessibility to the service, the quality of the service, and the quality of water for human consumption;
• The sustainability of the WUs, namely from an economic, infrastructural, and human resource perspective;
• Environmental sustainability, in the use of environmental resources, in circularity and valorisation, and in pollution prevention.

ERSAR’s strategy involves the following three major intervention domains:

• The sector’s structural regulation, by contributing to better organization of the sector and clarifying its rules;
• The behavioural regulation of the WUs throughout the life cycle in terms of legal and contractual monitoring, economic regulation, the quality of service provided, the quality of water for human consumption, and the interface with consumers;
• Complementary regulatory activities, which include the preparation and regular dissemination of information and technical support to the WUs.

Economic regulation and quality of service regulation complement each other and constitute forms of the behavioural regulation of the WUs. Both contribute to conditioning behaviour, with the regulation of the quality of the service being the regulation instrument that is best suited to conditioning the behaviour of the WUs, limiting them, or encouraging them to achieve certain service quality objectives.

It should be noted that it is up to ERSAR to define, on the one hand, minimum levels of quality for aspects that are directly related to the quality of the service provided to users and directly felt by them and, on the other hand, the compensation due in the case of non-compliance. This last aspect will correspond to a more intense level of regulatory intervention in terms of service quality.

In this context, the use of performance metrics, namely indicators or indices, is an essential tool for assessing the WUs in relation to specific aspects of the activity carried out. Performance indicators (PIs) express the level of quality of the service provided to the users through a quantitative measure of effectiveness and or efficiency, making a direct and transparent comparison between management objectives and obtained results [1]. The use of indicators to assess the quality of service constitutes an important regulatory instrument, standardizing the information collection and the performance assessment based on clear definitions and a common language in a way that is intended to be representative and balanced of the universe of WUs. For a clear understanding, indicators must be defined considering the relevant aspects of the service through an objective description, the assessment rule, and the respective reference values. Reference values are established thresholds allowing for a ranking of the performance indicator results with an associated judgment, which can be gradually adapted to drive improvement considering national strategic targets. The indication of data reliability is also of great importance for an adequate interpretation of the results.

The applied regulatory model in Continental Portugal thus includes the use of mechanisms for assessing the quality of the service provided to the users by each WU and for benchmarking with other similar WUs operating in different geographical areas. It is important to highlight that this assessment system applies to every water and waste service operator regardless of its activity scope, nature, management model, or size. By contributing to the quantification of the quality of the service, the use of performance indicators also allows for the comparative assessment of objective compliance and its analysis over time (evolution).

The assessment and benchmarking results are made publicly available, thus realizing a fundamental right that assists all users of these services. On the other hand, the publication of results encourages WUs to progress towards greater effectiveness and efficiency, as they naturally have ambition to be placed in a favourable position. This artificial competitive environment is a powerful regulatory tool within monopoly utilities, compelled to outperform their own or to achieve the same results as their peers. It is also intended to consolidate a true culture of information: concise, credible, and easy-to-interpret. Additionally, one of the main advantages proven during the application of this system of indicators refers to the requirement of the WUs to implement and keep up-to-date data collection and organization routines, imbedding this system of indicators in their own operation and service management. An assessment system needs to be parsimonious, particularly when designed for regulation purposes, not only to keep the focus on the essential aspects to assess regarding the quality of service, but also to ensure efficiency in the processes of data collection, PI calculation, and data validation and audit, considering the universe of WUs regulated.

Therefore, the assessment system for regulation should better use the shortest number possible of key performance indicators (KPIs), assessing the key aspects for the quality of service and ensuring they are applicable to all services and types of operators regulated, which bring additional challenges to the systems’ development.

The assessment of the quality of urban water services has been applied by ERSAR since 2004 when the first generation of the assessment system was developed with 20 KPIs for each water service, namely drinking water supply and wastewater management (coherent with an analogous system for the urban waste, also regulated by ERSAR). Following this first generation, three periodical critical revisions were conducted, each one originating a new generation of the assessment system.

From the accumulated experience and in view of the national strategic plans published in the meantime, it has become essential to evolve towards the establishment of a more adequate instrument for the assessment of all regulated WUs. The fourth generation was developed in 2021 and entered in force in 2022, with some of the new KPIs under testing, to evaluate the reference values proposed and the difficulty in collecting reliable information by the WUs. The first results will be available in late 2023.

This paper aims to present the fourth generation of ERSAR’s assessment system of the quality of the urban water services in Continental Portugal (QSAS-4G), focusing on the path followed and addressing the experience of application during almost two decades, the lessons learned, and the new challenges for the water sector and its regulation.

2. Materials and Methods

2.1. Global Approach

The approach followed for the development of the QSAS-4G (Figure 1), carried out in 2021, considered seven steps:

1.

Review and critical analysis of the third generation of the quality of service assessment system (QSAS-3G) applied in the period 2016–2020 [1] and the results of the respective application.

An analysis of the calculated values of the indicators for the previous four years (2016–2019) and of their reliability was carried out, aiming at identifying the quality of the information as well as the level of difficulty in collecting reliable data faced by the WUs. Dispersion of the WUs’ results and evolution over time of each indicator’s results were also analysed. This analysis allows for identifying the indicators with a low dispersion of the results reported by the WUs, either annual or over the years. Box and whisker charts and boxplots were used to visualise the results. Positioning and opening of the boxplots were analysed. In cases with low variability, the possibility of assessing national heterogeneity or capturing the evolution of the sector over time may be limited. Nevertheless, these indicators may still allow each WU to monitor individually its evolution over time. For this analysis, answers NA (not applicable) and NR (no reply) are not considered. Further analysis aimed to:

• Identify the indicators for which low variability occurs because the results are concentrated, generally mostly in the range of good performance. If the box-plot graph is similar over several years and presents values of percentiles P25, P50, and P75 relatively close (with reduced range of variation), then the indicator may no longer be relevant to assess the national evolution of the quality of service.
• Identify the indicators for which low variability occurs because a notable variation in the individual results is not perceptible. This can be visualized when the response is practically constant over time either for the WU that presents less inter-annual variability or for the WU with greater inter-annual variability. For this analysis, answers NA (not applicable) and NR (no reply) are not considered.

2.

Analysis of the national Portuguese strategic plan for the decade, for water, wastewater, and stormwater services—PENSAARP 2030 [2].

Even if this plan is in the formal approval process, already submitted to public consultation and planned to be approved during the course of 2023, an analysis of the new challenges addressed in PENSAARP 2030 (after the strategic plans for 2007–2013, 2013–2020) was carried out giving support to the literature review (Step 3).

3.

Literature review related to the new challenges for the water sector addressed in the national strategic plan, particularly related to the aspects mentioned below.

A literature review was carried out focused on: stormwater management; efficiency of services and circularity, e.g., energy efficiency and neutrality and water reuse; standardization in asset management and in assessment of the quality of urban water services; and cybersecurity and security against intrusion, as all these aspects will have impact on the WU service provision and, thus, on the respective quality of service.

4.

Definition of the priorities to be considered in the QSAS-4G, defined in line with PENSAARP 2030.

The alignment of the previous assessment system (QSAS-3G) with the national strategic plan was analysed in two stages. In the first stage, PENSAARP 2030 was considered as a starting point and its representativeness in the QSAS-3G was analysed, either reflected in the performance indicators or in the WU’s profile and characterisation metrics. The aspects of this strategic plan that were not contemplated in the QSAS-3G were identified. In the second stage, the starting point was the QSAS-3G, identifying which metrics of this system are considered in PENSAARP 2030. In this analysis, it was essential to consider the scope and purpose of the assessment in each instrument.

5.

Establishment of a proposal for the new performance assessment system, QSAS-4G, for water supply, wastewater management, and stormwater management services:

• Identification of the QSAS-3G metrics (indicators or indices) to be kept, reformulated, or eliminated considering the concept, definition, processing rule, reference values, results record (checking for possible stagnation or goal already achieved), and adequacy to the new structure (objectives and assessment criteria);
• Definition of the new assessment metrics (indicators or indices) or update of those to be changed;
• Applicability of the indicators according to the typology of the WU service (bulk or retail);
• Alignment with PENSAARP 2030;
• Definition of required data for calculating the performance assessment metrics (indicators or indices), including the corresponding reliability;
• Definition of data reliability and accuracy levels;
• Concept and definition of data for context information (WU and system profiles and other data considered relevant to ERSAR’s activity);
• Complementary definitions of the assessment system;
• Identification and evaluation of possible synergies in obtaining and uploading data from the information systems of other entities, such as the Portuguese Environment Agency and the Statistics Portugal, avoiding duplication of information reporting and validation.

Regarding the stormwater management service, the application scope, new indicators, and required data were defined from scratch, as there was no previous system for assessing the quality of this service in QSAS-3G. In this context, an analysis of the applicability of the current indicators of other water services to the stormwater management service was conducted.

6.

Disclosure and public consultation to the water and waste sector.

A seminar to present to the sector the proposal for the QSAS-4G was organised and held on 16 November 2021. This seminar involved interventions by ERSAR (president and technical team) and LNEC; all water and wastewater utilities regulated were invited and many actively participated. This event started the contradictory period (between 16 and 30 November 2021) for the WUs to submit to ERSAR their insights on the proposed QSAS-4G, providing to all WUs and water sector professional associations the opportunity to give their vision and contributions on the proposed QSAA-4G.

7.

Definition of the new performance assessment system, QSAS-4G, for water supply, wastewater management, and stormwater management services.

The information received from the consultation to the sector was analysed, and a final version of the QSAS-4G was presented.

2.2. Approach for Establishing Reference Values

The reference values translate the judgement of what is considered good, fair, and poor performance for each metric. They aimed to allow for identifying the main strengths and weaknesses regarding performance as well as comparing between cases (benchmarking) or monitoring the progress. For this reason, this judgement shall be established independently from the specific cases and be as stable as possible over time [3]. The reference values may be dependent on legislation. The reference values allow for the transformation of the performance values into performance levels.

Reference values may be established based on (Figure 2):

(1)

National or European legislation (mandatory if existing);

(2)

Regulation or standardization;

(3)

Strategic goals—National, European, and International strategic plans;

(4)

Theoretical concepts and technical requirements behind the metrics;

(5)

Literature reviews on best practices;

(6)

Statistical analysis of the metrics values associated to expert assessment of the cases (e.g., cluster analysis, percentiles distribution).

3. Results

This section presents the results achieved, detailing them in each one of the developed steps according to the global approach followed (Section 2.1): Step 1 regarding the review and critical analysis of the QSAS-3G; Step 2 with respect to the analysis of the strategic plan PENSAARP 2030; Step 3 focusing on the literature review related to the new challenges of PENSAARP 2030; Step 4 defining the priorities to be considered in the QSAS-4G, in line with PENSAARP 2030; and Steps 5 to 7 concerning the establishment of the new performance assessment system, QSAS-4G, for water supply, wastewater management, and stormwater management services, respectively.

3.1. Step 1|Review and Critical Analysis of the QSAS-3G

The ERSAR’s system to assess the quality of service for the water sector, QSAS-3G, in force from 2017 to 2020, is presented in

Table 1. QSAS-3G for water and wastewater services [1].

Objective	Indicator (Water Supply)		Indicator (Wastewater)
Adequacy of the interface with the user	AA01	Service coverage	AR01	Service coverage
	AA02	Affordability of the service	AR02	Affordability of the service
	AA03	Service interruptions	AR03	Flood occurrences
	AA04	Safe water
	AA05	Response to complaints and suggestions	AR04	Response to complaints and suggestions
Operator sustainability	AA06	Cost recovery ratio	AR05	Cost recovery ratio
	AA07	Connection to the service	AR06	Connection to the service
	AA08	Non-revenue water
	AA09	Mains rehabilitation	AR07	Sewer rehabilitation
	AA10	Mains failures	AR08	Sewer collapses
	AA11	Adequacy of human resources	AR09	Adequacy of human resources
Environmental sustainability	AA12	Real water losses
	AA13	Standardised energy consumption	AR10	Standardised energy consumption
			AR11	Accessibility to wastewater treatment
			AR12	Emergency discharges control
			AR13	Compliance with discharge permit
	AA14	Sludge disposal	AR14	Sludge disposal

. The formulation or the reference values of some of the indicators for WUs providing a bulk service (utilities a) is slightly different from the retail service (water distribution or wastewater collection, utilities b).

Based on ERSAR’s information from the WUs (yearly reports referring to the data from previous year: [4,5,6,7]), Figure 3 and Figure 4 present a summary of the WUs’ replies to the indicators for water supply (AA) and wastewater (AR) indicators, respectively. For each indicator and for the several years under analysis, the percentage of responses NA (not applicable) and NR (not answered) is indicated as well as for the set of available responses for the reliability (* reduced, ** intermediate or *** high) of the corresponding input data. As referred, the indicators are applied either to bulk WUs, identified with “a” (for example, AA01a), or to retail WUs, identified with “b” (for example, AR01b).

The intent of this graphical representation is the identification of the indicators for which a relevant percentage of WUs indicates it as not applicable, did not reply, or presented a response with reduced reliability. As mentioned, in these cases, the quantity and quality of collected information may be compromised. These indicators were flagged for analysis at a later stage regarding their adequacy to a given universe of WUs, the maturity of the information available, their relevance for the assessment system, or the need for a revision of the description of the indicator or of the input variables. In summary, the indicators AA01 and AR01, AA03 and AR03, AA07 and AR06, AA12, AA13b and AR10b, and AR11 and AR12b are highlighted.

The analysis of dispersion was carried out for all the indicators using box plots, as exemplified in Figure 5 for the water supply service. From such an analysis, the indicators with low variability at the national level are AA02ab—Affordability of the service, AA03b—Service interruptions, AA05b—Response to complaints and suggestions, AA06b—Cost recovery ratio, AA07b—Connection to the service, AA08b—Non-revenue water, and AA12b—Real water losses.

Regarding wastewater services, the indicators with low variability are AR02b—Affordability of the service, AR05b—Cost recovery ratio, AR06b—Connection to the service, AR09b—Adequacy of human resources, AR11ab—Accessibility to wastewater treatment, and AR14ab—Sludge disposal.

Still regarding the water supply service, the indicators that do not show individual variation are AA02a—Affordability of the service, AA03b—Service interruptions, and AA14a—Sludge disposal. Regarding wastewater, only AR14ab—Sludge disposal is highlighted.

As previously referred, these indicators were flagged for a later integrated analysis of the interest and benefit of (i) keeping the indicator in its current form; (ii) modifying its formulation, input variables, or reference values; or (iii) moving it to the system profile for characterization and for the sake of keeping the historical record.

3.2. Step 2| Analysis of the Strategic Plan PENSAARP 2030

The strategic vision is materialized in four global objectives in the strategic plan PENSAARP 2030 (final draft):

• Effective services that promote physical accessibility, continuity, and reliability; the quality of abstracted and discharged water, security, resilience, and climate action; equity; and economic accessibility;
• Efficient services that promote government and sector structuring, organization, modernization, and digitization of WUs; the management and allocation of financial resources; water efficiency and energy efficiency; and decarbonization;
• Sustainable services that promote economic and financial sustainability, infrastructural sustainability, the use and recovery of resources, human capital, and information management, knowledge, and innovation;
• Services valorisation, which promotes business and economic value in the internal and external markets, circularity, environmental and territorial valorisation, societal valorisation, transparency, accountability and ethics, and the contribution to sustainable development and international cooperation policy.

These global objectives break down into 20 specific objectives, which include (i) priority themes for their high importance or still unsatisfactory performance; (ii) other aspects that, despite their already high performance, must remain under the sector’s attention in terms of future sustainability; and (iii) aspects that constitute new challenges and opportunities.

It is important to analyse the strategic guidance proposed for the decade. PENSAARP 2030 [2] identifies several ways to pursue these objectives, as stated in

Table 2. Strategic guidance for the Portuguese water, wastewater, and stormwater sectors for the decade, as stated in PENSAARP 2030 [2].

Management Guidance
Improve articulation between water services and land management, aiming at a better general design of these systems and the infiltration or retention of stormwater in urban licensing, the need to integrate the costs of connections into overall costs, and the availability of support service in autonomous systems.
Improve the articulation between services and land use planning, namely in the development of green cities, urban rehabilitation, and hydrographic rehabilitation.
Define water infrastructure as critical infrastructure, a priority regarding its protection, requiring the existence of safety and contingency plans.
Improve the tariff structure, the criteria for subsidizing services, and the generalization of the social tariff to continue to ensure equity and economic accessibility of services to users, considering the pressure to increase tariffs for an effective recovery of efficient spending.
Recognize as urgent the adoption of a culture and practice of asset management of the infrastructures, applicable in large, medium, and small systems.
Strengthen the sector’s human capital and the consolidation of training programs.
Strengthen the systems’ security and resilience, with an emphasis on adapting services to climate change and other natural and human-induced risks, which may jeopardize the service provision.
Promote water efficiency by reducing water losses in water supply systems, reducing undue inflows to wastewater and stormwater systems, improving the macro measurement of supplied water, optimizing the macro measurement of wastewater and stormwater, and improving the water efficiency of household installations.
Promote resources management through (i) a more efficient water use; (ii) the use of alternative water sources, such as direct or indirect water reuse from wastewater treatment, desalination, and stormwater use; (iii) the proper management and valorisation of urban sludge and other by-products; and (iv) the selection and proper disposal of construction materials and components and of waste and hazardous waste used and produced throughout the life cycle of the systems.
Encourage energy efficiency improvement and the reinforcement of energy production allowing an evolution towards decarbonisation. It introduces various incentives, namely the need for WUs to have long-term planning and management instruments, articulated with the national strategic plan.
Gradually promote the circularity in infrastructures and associated services and their environmental enhancement, for example, relating to wastewater, sludge, and energy.
Infrastructural guidelines
Generalize the physical accessibility of the population to services through the completion of infrastructure, construction, and re-naturalization of stormwater infrastructure and the improvement of simplified public systems in rural areas.
Intensify the rehabilitation of the already existing infrastructure. This needs to ensure the continuity and reliability of the service in the long term for the improvement of the quality of the rejected/discharged wastewater and of the rejected/discharged stormwater.
Operational guidelines
Strengthen the environmental control by the WUs of the water abstraction and wastewater and stormwater discharge as well as of the articulation between WUs and the delimitation of environmental responsibility regarding discharges of industrial wastewater into urban wastewater networks and treatment infrastructures.
Promote the improvement of the inventory and operational knowledge of the infrastructures and the use of mechanisms for assessing infrastructural condition on a national scale and define the minimum content of the infrastructures register, including georeferenced collection and rejection points.

.

3.3. Step 3|Literature Review Related to the New Challenges of PENSAARP 2030

Regarding the stormwater management, generically, there is a general dispersion of responsibilities in the stormwater sector worldwide. Greater articulation between entities, clarification of the financial models, and the concern with environmental control are imperative. For example, cities such as Portland and Vancouver have been promoting the growth of green infrastructure as a solution for stormwater management associated with numerous ecosystem services. In Portland, incentives for solutions to control stormwater at the source along with a reduction in the service fee for urbanized areas were implemented. In Vancouver, a budget for the implementation, maintenance, and monitoring of green infrastructure was assigned [9]. The international literature points out the advantages of this green/blue approach to be implemented adaptively, progressively, and in a way that takes advantage of investment opportunities for systems’ expansion or rehabilitation [10,11,12,13,14,15,16].

The issue of the boundary of stormwater systems is relevant for assigning responsibilities and defining mechanisms for financing and controlling the service provision. Boundary delimitation should be defined not only upstream (where the contractual relationship with the customer is designed), but also downstream (where the requirements for environmental control and the protection of public health and safety are established). Stormwater systems are, at an international level, generically regulated by environmental regulatory bodies, with a greater focus on flood protection, on the protection of water bodies from pollution discharges, or on the increase in the cities’ resilience to climate variability and uncertainty. All these aspects are mostly focused on the downstream boundary. Regarding financing models for stormwater management, it should be noted that, in some countries, fees have been implemented, having been recognized as a successful mechanism to finance legal obligations and environmental protection. Other successful cases consider a combined system of fees and tariffs, and there are also situations where incentives are available for the adoption of solutions that reduce rainwater flows [17].

The European directive on urban wastewater treatment [18] is currently under review, and the new text [19] proposes the reduction of pollution due to rainwater in large agglomerations, which will require the implementation of integrated urban water management plans, where nature-based solutions are seen as good candidates for overflow treatment [10,11,12,13,14,15,16].

Looking at a circular economy in water services, along with the efficient use of resources (e.g., control of water losses and energy efficiency, aspects already evaluated in ERSAR’s quality-of-service assessment system—third generation), it aims to solve issues of global, regional, and local sustainability, and it is a topic of great relevance, with a prominent role in the policies and strategic plans of the sector, nationally and internationally. Internationally, there are several initiatives from financing entities, regulators, and associations in the water sector. The World Bank’s “Wastewater: From Waste to Resource” initiative [20] recognizes the potential of wastewater to create value and encourages a paradigm shift in wastewater systems, replacing traditional wastewater treatment plants (WWTPs) with resource recovery facilities, in particular, resources such as water, energy, biosolids, and fertilizers. The Finnish Institute for the Environment [21] highlights the circular economy on its research and development page and points out the four main areas of action, consensually considered the key areas for the sustainable use of water: reduction of water losses, reduction of inefficiencies in water use, recovery of substances and energy, and recycling and reuse of water. In turn, to support management bodies, regulators, financing entities, consultants, industry, and researchers, in 2016, the International Water Association (IWA) published a guide for the circular economy in the urban water cycle, summarized in three inter-related resources, namely water, materials, and energy [22]. The IWA proposes the urban water cycle to be thought of and managed as a closed (circular) system, with cascading water quality options, determined and differentiated for each use. To be successful, it considers that efficient reuse and transport systems are critical and that the recovery of resources must be competitive in a market driven by consumption. According to the IWA, the biggest challenge is finding markets willing to work with recovered products as alternatives to traditional products (manufactured or extracted). Production/recovery scale and consumer acceptance are identified as key issues. For the IWA, the energy portfolio should aim to reduce fossil energy consumption, increase renewable energy production and consumption, and contribute to zero carbon cities [22].

EurEau, the European Federation of National Water Services Associations, has identified 10 major challenges for the water sector over the next 10 years, one of them being “giving water its value in the circular economy”. Wastewater (and its by-products, e.g., sludge) contain valuable resources, such as energy and phosphorus, that can be recovered and reused to save scarce or depleted resources (and the associated negative impacts) and to promote economic growth and job creation. Jobs and incentives should be created to channel the recovered resources to the market in a sustainable way [23].

The new proposal for the EU urban wastewater treatment directive [24] calls for the WUs to systematically consider water reuse, reflecting the advances achieved in the meantime. As for energy, it proposes the energy neutrality of the urban wastewater treatment. It is in the circular economy context that WWTPs should be considered resource recovery facilities for water, energy, and materials, such as phosphorus.

On the topic of standardization in asset management and in assessment of the quality of urban water services, ISO 55000 [25] provides an overview of asset management and of management systems for asset management. This standard frames the principles and terminology of asset management and the benefits expected from its adoption. It defines asset management as the coordinated activity of an organization to realize and produce value from assets. It also specifies the context for the ISO 55001 [26] and ISO 55002 [27] standards. The ISO 55001 standard specifies the requirements for an asset management system, and ISO 55002 provides guidance on how ISO 55001 should be interpreted and applied, focusing on the specific requirements for asset management, the asset management system, and the asset portfolio. Urban water systems are part of a vast portfolio of physical and other assets, including the infrastructure that provides the service. Although asset management is broader in terms of the topics it encompasses, infrastructure asset management (IAM) applied to urban water systems goes beyond the management of infrastructures as assets themselves and contemplates the specificities of the infrastructures in question, namely their interdependencies and the cause–effect relationship of their behaviour. IAM also intends to ensure the sustainability of these assets in the long term, ensuring the service’s objectives and the respective targets over an extended horizon and considering all dimensions of sustainability: social, environmental, and economic. Complementarily, some of the ISO 2451X standards focus on the management of the water service assets. These standards were considered less relevant than the ISO 5500X in the context of the QSAS-4G.

It is considered that these aspects are safeguarded in ERSAR’s assessment system. Additionally, as the third generation, the system to assess the quality of service, QSAS-3G, was developed in line with ISO 24510 [28], ISO 24511 [29], and service ISO 24512 [30] standards (currently being reviewed, with no major changes expected); it is therefore considered that its structure remains updated.

With regard to cybersecurity and security against intrusion, it is fundamental to recognize that there are more and more actors with the potential for connection and an eventual motivation for intrusion. These motivations can be financial, economic, political, criminal, or terrorist, among others [31]. In this respect, the existence of security systems against intrusion, namely to prevent clandestine activities in reservoirs and catchment areas, is already recommended within the scope of water safety plans [32]. The European Directive 2008/114/EC (ECI Directive) [19] considers that each Member State must identify potential critical infrastructures, which are essential for maintaining vital functions for society, health, safety, and economic or social well-being and whose disruption or destruction would have a significant impact on a Member State. The energy and transport sectors are supposed to be identified, but others may be included if deemed appropriate. This directive, laying down obligations on the identification and designation of European critical infrastructure, was recently revised and published, introducing additional critical infrastructure protection measures [33]. ERSAR intends to develop a regulation and complementary technical instructions for the water sector to define safety requirements and report incidents. A technical guide [34] is available, which deals with the communication plan of emergency(s) in the quality of water for human consumption. In this context, the following are considered as incidents: sabotage; bioterrorism; cyberterrorism; vandalism; spillage of hazardous chemicals; and fire. These may be adapted to urban drainage systems.

3.4. Step 4|Definition of the Priorities to Be Considered in the QSAS-4G, Defined in Line with PENSAARP 2030

From the analysis of the alignment of the PENSAARP 2030 with the QSAS-3G, relative to the first stage described above (Section 2), it is possible to identify that the structure of the general and specific objectives of PENSAARP 2030 is different from that of QSAS-3G, this difference arising from the particular scope and purpose of each system. Generically, each objective of PENSAARP 2030 is reflected in several objectives of QSAS-3G, with no two-way relationship. Additionally, stormwater management is considered in PENSAARP 2030 and not in QSAS-3G, as it is not part of the scope of regulation.

For most of the specific objectives of PENSAARP 2030, assessment metrics, profile metrics, or variables are available in QSAS-3G. This aspect is not verified for four specific objectives: C5—Management of information, knowledge, and innovation; D1—Business and economic enhancement; D4—Transparency, accountability, and ethics; and D5—Contribution to sustainable development. For these specific purposes, there is no alignment between the two systems. It was considered that they do not need to be incorporated in QSAS-4G since they do not fall within the scope of the regulatory assessment of the quality of service.

From the results of the alignment analysis of QSAS-3G with PENSAARP 2030, relative to the second stage described above (Section 2), it is possible to identify that, despite the different structure of the objectives between the two assessment systems, almost all metrics of the QSAS-3G are included in PENSAARP 2030 objectives. The exceptions are two metrics, AA14 and AR14 (adequate destination of treatment sludge), since the strategic plan considers an evolution towards the sludge valorisation/beneficial use as a contribution to circularity and to environmental and territorial valorisation.

Additionally, regarding stormwater management, there is great potential for aligning the current QSAS-3G objectives with PENSAARP 2030. Fifteen metrics are identified in PENSAARP 2030 relating to stormwater that are not currently part of QSAS-3G.

3.5. Steps 5 to 7|Establishment of a New Performance Assessment System QSAS-4G for Water Supply, Wastewater Management, and Stormwater Management Services: First Proposal, Public Consultation to the Sector and Final System

QSAS-4G was developed, defining the objectives, assessment criteria, and performance metrics. This system also included the description of the metrics’ concept, definition, required data, and the levels of reliability and data accuracy. Other descriptions were developed: the concept and definition of data used for context information (WU’s and system’s profiles and other information considered relevant to ERSAR’s activity); complementary definitions of the assessment system; and identification and evaluation of possible synergies in data collection and uploading.

For the key aspects of service provision that currently do not have the possibility of being qualified due to the difficulty in assigning reference values without any baseline information or requirements, metrics were considered within the systems’ profile, or as indicators under testing, until it is possible to evaluate them. Performance indices were also considered, which aggregate information regarding relevant aspects about the services. The subdivision of indicators was avoided, and an attempt was made to simplify and minimize the amount of information requested from the WUs. To address the new challenges and the future perspectives for urban water services’ assessment in the near future (5 years), complementary information, considered as essential to be collected from the WUs, was identified and codified.

Regarding the stormwater management service, as it constitutes a new assessment system, it was necessary to define the scope of the assessment considering the current maturity in the management of this service, the alignment with the assessment systems of the other water services, and the alignment with PENSARP 2030. The decision was, at this stage, to define and ask for key input variables instead of launching a comprehensive assessment system. Only a new indicator was included.

A public consultation to the sector was carried out (Step 6) involving all the WUs regulated as well as the most relevant Portuguese technical associations of the regulated sector, consumer organizations, Portuguese Universities, and associations representing all ERSAR’s regulated utilities and the Advisory Board of ERSAR.

ERSAR received 650 contributions regarding the proposed QSAS-4G. All contributions were analysed, in some cases leading to the reanalysis of some data, indicators, and concepts; when necessary, changes were included in the final version of the QSAS-4G.

In addition, the public consultation report, encompassing all the answers to the contributions, was publicly released on ERSAR’s website. The final version of the QSAS-4G assessment system is presented in

Table 3. QSAS-4G for water and wastewater services [1] (new or refined indicators are highlighted in bold).

Objective	Criterion and Indicator (Water Supply)		Criterion and Indicator (Wastewater)
Adequacy of the service to the user	Accessibility of the service to the users
	AA01	Service coverage (%)	AR01	Service coverage (%)
	AA02	Affordability of the service (%)	AR02	Service coverage through network and septic tanks (%)
			AR03	Affordability of the service (%)
	Quality of the service provided to the users
	AA03	Service interruptions [number/(delivery point·year)] Service interruptions [number/(1000 service connections·year)]	AR04	Flood occurrences [number/(100 km·year)] Flood occurrences [number/(1000 service connections·year)]
	AA04	Safe water (%)
	AA05	Response to complaints, suggestions, and information requests (%)	AR05	Response to complaints, suggestions, and information requests (%)
Operator sustainability	Economic sustainability
	AA06	Cost recovery (%)	AR06	Cost recovery (%)
	AA07	Connection to the service (%)	AR07	Connection to the service (%)
	AA08	Non-revenue water (%)	AR08	Connection to the service through network (%)
	Infrastructural sustainability
	AA09	Mains rehabilitation (%/year)	AR09	Sewer rehabilitation (%/year)
	AA10	Mains failures [number/(100 km·year)]	AR10	Sewer collapses [number/(100 km·year)]
	AA11	Adequacy of treatment capacity use (%)	AR11	Sewer pipes condition monitoring (%)
			AR12	Adequacy of treatment capacity use (%)
	Physical productivity of human resources
	AA12	Adequacy of human resources in adduction and treatment [number/(106 m3·year)]	AR13	Adequacy of human resources in transport and treatment [number/(106 m3·year)]
	AA13	Adequacy of human resources in water treatment [number/(106 m3·year)]	AR14	Adequacy of human resources in wastewater treatment [number/(106 m3·year)]
	AA14	Adequacy of human resources in water distribution [number/(1000 service connections·year)]	AR15	Adequacy of human resources in wastewater collection and drainage [number/(100 km·year)]
Environmental sustainability	Efficiency in the use of environmental resources
	AA15	Real water losses [m3/(km·day)] Real water losses [L/(water connection·day)]	AR16	Energy efficiency of pumping facilities [kWh/(m3·100 m)]
	AA16	Energy efficiency of pumping facilities [kWh/(m3·100 m)]	AR17	Treatment sludge production (kg/m3)
	AA17	Treatment sludge production [kg/m3]
	Circularity and recovery
	AA18	Energy self-sufficiency (%)	AR18	Reclaimed water production (%)
			AR19	Energy self-sufficiency (%)
	Efficiency in pollution prevention
			AR20	Emergency and stormwater discharges control (%)
			AR21	Compliance with discharge permit (%)

, where the new or refined indicators relative to QSAS-3G, are highlighted in bold.

The safe water metric (AA04) is the result obtained from the implementation of the quality control programmes throughout the distribution system, previously approved by ERSAR, the national authority for drinking water quality. It includes all laboratorial analytical data results of these programmes for one year and is defined as the percentage of tests carried out among those required and that complied with the parametric values.

All definitions and formulas of the KPIs in Table 3 can be consulted in Cardoso et al. [1].

The reference values were established following the approach presented in Figure 2. Figure 6 demonstrates this rationale and exemplifies it for the energy self-sufficiency metric (AR19).

The assessment of the quality of the stormwater management service was introduced, developing the AP01 indicator defined to assess the level of knowledge on the infrastructures of the separate stormwater system. This indicator aims to assess the sustainability level in the service management in terms of infrastructural sustainability, concerning the operator’s existing knowledge of the stormwater infrastructures in its area of intervention, which is considered critical since it is the first step in the efficient management of this service. The following classes are assessed:

• Class A—Existence of infrastructure mapping;
• Class B—Information recorded on the separate sewers and wastewater connections;
• Class C—Information recorded on the remaining infrastructures;
• Class D—Information recorded on the metering equipment;
• Class E—Information recorded on the condition of the infrastructures;
• Class F—Information recorded on interventions in the public network;
• Class G—Interconnection between the geographic information system and other information systems of the operator and the recording of risk factors.

This is determined by the sum of the grades of each class under analysis, with a predefined number of points being assigned to each question, which may vary from 0 to 200 points.

4. Discussion

The transition from the third to the fourth generation of the Portuguese regulator’s assessment system of the urban water service’s quality posed the challenges of ensuring continuity (historical series), learning from the experience of the previous applications, keeping the size of the assessment system small (by selecting the critical aspects only), and, simultaneously, responding to pending or new issues, in line with the national strategy for the sector and the foreseen EU policy. The recent proposal for the new Urban Wastewater Treatment Directive (UWWTD) [24] reflects the requirements for an integrated management of wastewater and stormwater, introducing the obligation to establish locally integrated urban wastewater management plans to combat pollution from stormwater (urban runoff and stormwater overflow). It also includes the requirements for a circular economy with tracking at the source, systematic water reuse, and sludge with waste hierarchy. Additionally, the proposal changes the scope to include smaller towns (1000–2000 inhabitants), standards are reinforced for N and P removal, the scope of facility application is enlarged (above 100,000 p.e.), and new emission standards are related to micropollutants, introducing the responsibility concept extended to the producer. Regarding energy and climate, it considers the achievement of energy neutrality by 2040 at the national level in all treatment facilities above 10,000 p.e. and the monitoring of greenhouse gas (GHG) emissions. A new obligation is included to assess the risks caused by urban wastewater discharges to the environment and human health and, where necessary, to take additional measures on top of this directive’s minimum requirements to address these risks. Thus, even though the fourth generation of the assessment system was developed before the latest updates of the new UWWTD, it is aligned with them in the following aspects:

• Integrated wastewater and stormwater management through the indicators for the wastewater management service and the inclusion of an indicator related to the level of knowledge on the infrastructures of the separate stormwater system (the last one was not applied in 2022);
• Addressing smaller settlements, including an indicator translating the service coverage through network and septic tanks (AR02);
• Energy neutrality by considering the indicators energy efficiency of pumping facilities (AR16) and energy self-sufficiency (AR19);
• Water reuse by assessing the indicator reclaimed water production (AR18) and defining reference values based on the local water scarcity, expressed by the water exploitation index plus (WEI+);
• Sludge/biosolids/byproducts valorisation/beneficial use, including the indicator treatment sludge production (AR17);
• Reinforcement of N and P removal, related to the indicator compliance with discharge permit (AR21).

The fourth generation of the assessment system responds to changes in context and new challenges established by the national strategic plan, by the applicable strategic legislation (national and international, in force or under revision), and by adapting to the best practices available at an international level, namely those defined by the applicable CEN and ISO standards, among other reference documents. The attention paid to stormwater systems, safety management, the circular economy (e.g., water reuse, energy efficiency, and self-sufficiency towards energy neutrality), and, more generally, the concerns associated with the impacts of climate change on these services stand out. A set of objectives and criteria was identified (Table 3) that aims to promote that the services provided to users are adequate and sustainable and that they correspond to environmentally sound practices with resources’ recovery and valorisation. A structure of objectives and criteria that was as similar as possible to that of SAQS 3G, presented in Table 1, was kept, also considering parsimony in the set of indicators per service. There were, however, some criteria whose analysis was intended to be more developed, particularly those related to circularity and valorisation, but which, due to a lack of data or sector’s maturity, were not possible to introduce in SAQS 4G.

The reference values were critically analysed. For those indicators with minor changes or without changes, the results from Step 1 as well as the comments from the WUs (collected during the application over the years and the public consultation/contradictory phase) and information from the literature review, including PENSAARP 2030, were considered. For the new indicators, the approach for establishing the reference values (Section 2.2) was followed, and a sensitivity analysis was carried out whenever possible. For instance, the reference values of KPI AR17—Sludge production were changed after the public consultation to become a function of the WWTP inflow characteristics since the sludge production increases with the organic load, well represented by the influent chemical oxygen demand (COD), as shown by the sensitivity analysis carried out with WU field data (Figure 7).

5. Conclusions

The experience acquired during the application of the first generation of the assessment system, strengthened by the extension of the application of the second generation to all operators in Continental Portugal, and the publication of International Standards [28,29,30], relative to water services’ assessment, provided important contributions to the improvement materialised in the third generation. The accumulated experience since the application of the first generation, the reported WUs’ difficulties and improvement proposals, the publication of the ISO 5500X standards on asset management, and the new challenges identified in the new Portuguese Strategic Plan for the Water Supply and Wastewater and Stormwater Management Sector 2021–2030 (PENSAARP 2030) have contributed significantly to a guided reflection. This led to an improvement and consolidation of the assessment system of the quality of the water services currently regulated, i.e., water supply and wastewater management, and the recognition of the need for greater knowledge on the stormwater management service.

Many aspects of the assessment system have been improved based on all the experience gained over the almost 20 years of application of the first, second, and third generations of the assessment system. Nevertheless, during this revision process, care was taken to prevent major changes in the structure of the data collection that the service quality assessment process requires from the operators. Accordingly, clarifications were added, and for the three services overall, eleven KPIs were further refined, seven were eliminated to simplify the system, and eleven new indicators were included when considered pertinent and suitable to the goals of the assessment. To a large extent, these changes reflect the sensitivity acquired during intense contact with the operators, the changing context, the evolution of the sector and legislation, good practices, and international recommendations.

The application of the fourth generation of the system to assess the quality of service will continue to contribute not only to protect the interests of the users, but also to safeguard the interests of the regulated operators and of the entire sector in general.