Next Article in Journal
Assessing Corporate Vendor Selection in the Oil and Gas Industry: A Review of Green Strategies and Carbon Reduction Options
Previous Article in Journal
Driver Adaptability When Traffic Side Is Switched from Left to Right and Vice Versa: A Driving Simulator Study with Chinese and Pakistani Drivers
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Assessing the Techno-Economic Feasibility of Waste Electric and Electronic Equipment Treatment Plant: A Multi-Decisional Modeling Approach

Polytechnic Department of Engineering and Architecture (DPIA), University of Udine, Via delle Scienze 208, 33100 Udine, Italy
*
Author to whom correspondence should be addressed.
Sustainability 2023, 15(23), 16248; https://doi.org/10.3390/su152316248
Submission received: 17 October 2023 / Revised: 13 November 2023 / Accepted: 22 November 2023 / Published: 23 November 2023

Abstract

:
Nowadays, sustainable approaches to waste management are becoming critical, due to increased generation and complex physicochemical composition. Waste electric and electronic equipment (WEEE) management, in particular, is being given increasing attention due to the continuous augment in electronic equipment usage and the limited recycling rates. In this work, a multi-objective engineering optimization approach using a decision support system (DSS) was used to analyze the feasibility of installing a WEEE treatment plant in the Friuli-Venezia Giulia region (Northeastern Italy), considering that most of the produced WEEE is currently exported outside the region. Meaningful economic and environmental parameters were considered in the assessment, together with current WEEE production and composition. Plant investment cost was in the range of EUR 7–35 M for a potentiality of 8000–40,000 ton of treated WEEE/yr, the lower bound corresponding to the WEEE produced in the region. Payback time was 4.3–10 yr, strongly depending on the market’s economic conditions as well as on plant potentiality. Proper public subsidies should be provided for a plant treating only the locally produced WEEE, establishing a circular economy. The fraction of recovered materials was 78–83%, fulfilling the current EU legislative requirements of 80% and stabilizing around values of 80% for a higher washing machine fraction. An increase in personal computers may allow to augment the economic revenues, due to the high conferral fees, while it reduces the amounts of recovered materials, due to their complex composition. CO2 emission reduction thanks to material recovery was in the range of 8000–38,000 ton CO2/yr, linearly depending on the plant potentiality. The developed DSS system could be used both by public authorities and private companies to preliminarily evaluate the most important technical, financial and environmental aspects to assess overall plant sustainability. The proposed approach can be exported to different locations and integrated with energy recovery (i.e., incineration of the non-recoverable fractions), analyzing both environmental and economic aspects flexibly.

1. Introduction

Global population increase and economic growth unavoidably lead to increased waste generation, which poses significant sustainability challenges, especially if the waste is improperly managed [1]. Source selection of the generated waste allows to recover valuable resources and/or energy through the development of the so-called “biorefineries”, contributing to sustainable waste utilization, with a reduction in environmental pollution and additional economic benefits [2]. Given the raising interest in urgent environmental issues, including climate change and natural resources depletion, the European Union (EU) is strongly encouraging the transition from linear economy to a more sustainable circular economy (CE) framework in waste management [3]. CE consists of a closed-loop system, where a zero-waste philosophy is applied, with a holistic approach, aiming at a regenerative system, in which both waste and input energy flows are minimized [4]. The CE paradigm attracted increasing interest in the last years, following the most recent EU guidelines on waste management [5,6].
Among the different waste fluxes, waste from electric and electronic equipment (WEEE) stands out as the fastest-growing waste stream [7]; in addition, WEEE management faces significant technological challenges, due to the contemporary presence of recyclable and hazardous components [8]. Considering quantitative data, yearly, Oceania generates 17.3 kg WEEE/inhab, while Europe and America follow with, respectively, 16.6 kg/inhab and 11.6 kg/inhab of produced WEEE [9]. Global WEEE generation reached 53 Mt in 2019 from 33.8 Mt in 2010 [10], and a further 3–4% increase is expected annually in the next years [11,12].
In the market, nearly 900 different categories of electrical and electronic equipment (EEE) are currently present, including information and communication technology (ICT) products, such as personal computers and mobile phones, video cameras, and household equipment (e.g., washing machines), refrigerators and air conditioners [13,14]. Amongst WEEE, mobile phones and computers stand out as the most common wastes, due to their short life-span, widespread usage, and rapid development and replacement [6]. WEEE, also called e-waste, which originates from discarded EEE, is composed of up to 69 elements (precious and base metals, critical raw materials) [14], and contains costly components with a significant economic value, if properly recycled and recovered [15]. However, WEEE also includes a large number of hazardous compounds (mercury, brominated flame retardants, chlorofluorocarbons—CFC, and hydrochlorofluorocarbons—HCFC) that, if improperly managed, can lead to heavy environmental pollution, as well as human health risks [14,15,16].
When compared to other waste streams, the correct management and recycling of WEEE are particularly critical due to the peculiar waste composition and continuously increasing WEEE amounts. Also, landfill conferral is still the prevalent disposal method for all fractions that are not precious or base metals, with negative environmental burdens [14]. A recent study focused on two Scandinavian countries, i.e., Denmark and Norway, showed that there is still a limited degree of circularity in WEEE management, with a mismatch between companies’ approach to end-of-life management and the ambitions of the European WEEE Directive (2012/19/EU). This is even more surprising if considering that the Northern EU countries currently show the best performances in WEEE recycling, with values above 60% in Sweden and Switzerland, and between 38 and 50% in Norway and Denmark [17]. In the 2012 WEEE Directive, the EU imposed strict recycling quotas for the different WEEE categories [18], differentiated for each State Member; remarkably, these quotas cannot be fulfilled only by metal and glass fractions, but also plastic recycling is needed, due to its significant content (20–35%) in most WEEE [19].
As previously highlighted, WEEE management has become a global concern, due to rapid economic growth, urbanization, and growing high-tech demand [20], together with decreasing product service life [21]. However, only about 20% of all the generated WEEE is currently documented to be correctly collected and recycled [10], while a prevalent fraction of the produced WEEE is still incorrectly disposed of, even in illegal ways [22]. Still, an unfair, uncircular, unsustainable, and unequal WEEE market towards African countries (e.g., Nigeria) exists, which negatively impacts both the environment and human health [23]. Even in advanced countries like Germany, the mass of collected WEEE is significantly lower than the amount of EEE put on the market, so the recycling quotas required by the EU legislation are still not satisfied [24]. Some critical aspects that need to be addressed to improve global WEEE management may include a mandated network registry, the integration of formal and informal sectors, better consumer awareness and improved eco-designs, further investments in recycling and reuse facilities, improved disposal facilities, regulated transboundary movements, manufacturers’ responsibility, and a more stringent law enforcement [9].
Thus, WEEE recycling and resource recovery clearly represent an opportunity to reduce greenhouse gas (GHG) emissions and global environmental impacts [25]. Furthermore, as previously introduced, e-waste recycling can be an outstanding source of raw materials (including rare earth metals) for European economies, which traditionally lack mineral resources; interestingly, recycling processes allow for reducing both energy usage and CO2 emissions when compared to raw ore processing [26]. The feasibility of WEEE reuse, instead, strongly depends on commercial alternatives for product application, with power consumption and life-span being major factors in the overall reuse feasibility [27]. Remarkably, not always WEEE reuse is environmentally more sustainable than recycling, especially when dealing with energy-intensive equipment (e.g., white goods), whose efficiency throughout their life-span is more impactful than the construction phase [28]. Reuse of small electric devices, instead, is typically more sustainable than recycling since product utilization is less impactful than manufacturing [28].
Due to the interaction and contradiction between different economic and environmental objectives, the decision-making process can be extremely complicated when working with real complex wastewater or waste flows, such as WEEE [29,30]. Innovative approaches for WEEE management (including modeling tools focused on material flow analysis and life cycle assessment—LCA) are being currently developed in the literature to enhance overall sustainability, with a focus on resource recovery [31,32]. In particular, multi-objective optimization is useful to evaluate complex scenarios, including WEEE management, providing useful insight to decision-makers concerning the overall sustainability of alternative solutions for WEEE treatment. A multi-objective optimization approach was recently applied to study green chain network design and sustainable distribution systems [33]. Similarly, a multi-objective logistic model, including economic, environmental, and social factors was proposed in [34] to optimize reverse logistics in the Indian WEEE market: supply chain profitability, environmental impact of processes and activities, and social benefits were thoroughly analyzed [34]. However, to the best of the authors’ knowledge, a thorough approach to preliminarily evaluate the techno-economic sustainability and environmental benefits of a WEEE treatment plant is currently missing in the literature.
The present research was aimed at developing a decision-support tool to preliminarily assess the financial sustainability and related environmental burdens of a WEEE treatment plant, providing a useful tool for decision -makers (public authorities and private companies). The case study of the Friuli-Venezia Giulia region (Northeast of Italy) was specifically investigated, given the lack of dedicated plants on the territory, but the proposed approach can be exported to any location by properly specifying the related boundary conditions (amount of produced WEEE, specific WEEE composition, conferral fees, economic subsidies for the recovered products). The different WEEE fractions were investigated, together with their specific composition, to evaluate the total amount of resources that could be recovered. A plant layout treating the produced WEEE was designed, considering the specificity of the different WEEE fractions. The economic sustainability of the investment was assessed considering actual market conditions for WEEE, together with plant operating costs. The environmental impact was analyzed with CO2 emissions reduction thanks to resource recovery, as well as considering the fraction of recovered materials. A multi-objective optimization model (modeFRONTIER® 2021R3) was employed to evaluate the variation of meaningful economic and environmental parameters under changeable input conditions, including plant potentiality, WEEE-specific composition, conferral fees of the different WEEE fractions, and incomes given by national packaging consortium. Parameter variation was consistent with current market evolution. A wider utilization of standardized mathematical tools such as the present one to thoroughly evaluate the techno-economic and environmental sustainability of a WEEE treatment plant may allow to stimulate in the future increased WEEE recycling with valuable resource recovery, highlighting the advantages of conscious and correct management of this important waste stream.

2. Multi-Decisional Modeling Approach

Given the level of detail of the information available from the regional waste management databases (i.e., ARPA) [35] used for gathering input data for the model, the work has been structured in sequential steps, as schematized in Figure 1. The treatment capacity of the WEEE plant considered regional waste availability to enhance the environmental sustainability of WEEE management from a circular economy perspective.
A state-of-the-art review of decision-making methods applied to sustainable energy planning was provided in [36]. The investigation phase forecasts data acquisition (WEEE production and availability, product composition, and feasible recovery technologies). The design phase consists of scenario analyses, performed using mathematical simulations; the first step of this phase is the definition (through the so-called formative scenario analysis) of the different WEEE products to be recovered, starting from a list of available waste products on public databases. The specific features of each product, such as recoverable material fractions, must be known, including the presence of hazardous materials or fluids that need particular disposal procedures. Moreover, the plant’s technologies and economic features, such as capacity-specific capital costs, but also operations and maintenance (O&M) costs, must be available to allow a thorough assessment.
Besides technical constraints for each considered WEEE product, territorial boundary conditions were also considered, e.g., the local waste availability, automatically excluding from the DSS evaluation the non-compliant scenarios. Finally, concerning the constraints imposed on the model outputs, compliance with some threshold values of economic indicators (e.g., the maximum payback time) was imposed to exclude economically inconvenient scenarios.
Since the aim of the toolbox was a sustainability evaluation of local WEEE recovery and treatment, besides the economic objective functions, the environmental (minimization of carbon dioxide emissions) and the energy (maximization of primary energy saving) aspects were considered as well within the developed DSS.
The decision variables considered within the DSS for the model development, which are explained in detail in the Results section, are briefly: (i) the treatment plant capacity; (ii) the mass of recoverable materials; (iii) the economic value of these components; and (iv) the value of the economic subsidies linked to the environmental and energy benefits, such as the Italian energy efficiency stocks (TEE), that correspond to one ton oil equivalent (TOE) of primary energy saving. The investigation was carried out by varying the decision variables in a pre-defined range with a chosen step of variation.
The four-step simulation procedure (design of experiment—DOE, calculation, initial population variation) of the multi-objective optimization model was conducted following the procedure reported in [37] with the support of a commercial software (modeFRONTIER® 2021R3).
The last phase consists of decisionmaking, performed using a critical comparison of the performances of the different scenarios and an analysis of the impact of the most important decision variables on the objective functions.

3. Treatment Plant Modeling

The main aim of the conducted research was to reuse the recovered material in the analyzed regional area, where several energy-intensive industrial plants are present (e.g., steel casting facilities).

3.1. Waste Availability and Composition

As previously reported, quantitative data regarding WEEE production in the analyzed region were obtained from ARPA’s (i.e., the Regional Agency for Environment Protection) public database [35]. WEEE characterization, in terms of single product categories, was conducted following the actual WEEE market and availability as well as with interviews with experts in the field. For each category, different commercial products were considered, analyzing the possible materials to be recovered, including ferrous (steel, iron) and non-ferrous (aluminum, copper, brass) metals, as well as other important materials such as glass, oil, concrete and recoverable plastics, and still-working ancillary components (engines, pumps).
The non-recoverable fractions, including thermosetting plastics and rubbers, electric components, and fluids, that need special treatment to avoid negative environmental burdens and/or hazardous effects originating from an uncontrolled dispersion, such as CFCs, were supposed to be sent to a dedicated disposal. The detailed composition of each WEEE stream obtained from this preliminary assessment, together with the recoverable and non-recoverable fractions from each WEEE category, is summarized in Figure 2.
The treatment operations of the collected WEEE are described more in detail in the following section.

3.2. Plant Design

The considered plant layout (Figure 3) is essentially composed of (i) a storage and an intake system; (ii) a pre-treatment phase, where all the hazardous materials and fluids (cathode tubes from monitors and televisions, CFC refrigerants from refrigerators) are manually separated; (iii) a two-step grinding phase (where all the waste components are dimensionally reduced to facilitate the following separation); and (iv) a selection phase, where the materials are divided into recoverable and non-recoverable fractions for successive valorization and/or dedicated treatments.
Conventional separation techniques (magnetic and eddy current separators, grinding, and sieving) are used to recover the metals, separating them from the other materials, mostly involving plastics, inert fractions (glass, concrete, ceramics), and wood. The separated materials (especially metals and selected plastics) can be then recovered and valorized, or disposed of if their quality is too low (e.g., mixed plastics).
The treatment plant was schematized using a simplified model developed in Microsoft Excel (MS Office LTSC Professional 2021)®, reporting mass and energy balances for each treatment step. Considering the total weight of each WEEE product as Qi (ton/yr) and the specific material fraction that makes up the product as xi, the total amount of recovered materials (Qm, ton/yr) can be expressed as:
Q m = i x i · Q i
where the subscript i represents the considered product category.
Multiplying the results of Equation (1) for the specific emission factor of the considered material fraction (μ, ton CO2,eq/ton), the expression for GHG emission reduction (ton CO2,eq/yr) obtained from the recovery of the analyzed fraction can be easily obtained:
C O 2 = i x i · Q i μ
Extending the sum to each recovered material fraction, the overall GHG emission reduction (ton CO2,eq/yr) due to resource recovery can be finally expressed as:
C O 2 = j i x i j · Q i μ j
where the subscript j represents the material fraction.
The electricity request for the plant’s operation considers the global request of all the separation and treatment phases. Power functions of the selected process technologies were built as proposed in [38] and reported in Equation (4):
P y = P r e f · x y x r e f α
where the electricity request of the considered technology, Py (kW) at a defined capacity xy (ton/yr) is determined knowing its reference power request (Pref, kW) at a known reference capacity xref (ton/yr), and by applying a scale factor α.
As concerns the thermal energy request, it was necessary to cover the plant’s thermal needs; the solution able to minimize GHG (CO2, CH4) and other pollutant (NOx) emissions was chosen to limit the overall environmental impacts as much as possible: following a preliminary technical analysis, a gas-fired boiler was selected.

3.3. Economic Analysis

To thoroughly evaluate the plant’s performances, economic aspects were also taken into consideration in the techno-economic assessment. Plant investment costs include storage areas, sheds, canopies, and offices, where a unit cost expressed in EUR/m2 was considered, as well as all electromechanical devices for WEEE dismantling and separation, and trucks for waste collection and transportation. For electromechanical equipment, the cost of commercial devices was used, while collection and transportation costs were estimated using interviews with experts in the field. The considered operating costs forecast electricity and natural gas, and fuel for trucks, as well as the maintenance of the infrastructure and electromechanical equipment. The revenues are connected to two different aspects: (i) the input conferral fees of the different WEEE fractions, expressed as EUR/ton, multiplied by the mass of each WEEE stream; (ii) the subsidies provided by the national packaging consortium (CONAI) for the recovered materials, according to the specific composition of each WEEE stream.
As explained in the previous sections, the plant was composed of specific processes operating in different phases in the process scheme. Cost functions of the applied technologies were built by applying Equation (4) and by substituting power with costs, expressed as EUR.
The revenues deriving from material recovery were obtained by substituting the emission factor μ in Equation (3) with the specific cost of each material fraction (expressed as EUR/ton), leading to the following equation:
R m = j i x i j · Q i c j
Economic analysis was carried out through the calculation of the initial capital cost and annual cash flows. Two common performance parameters were utilized to compare various scenarios: net present value (NPV) and payback time (PB). They were calculated as follows:
N P V = C T O T + y = 0 15 F C y 1 + k d i s c o u n t + i n f l y
P B = C T O T F C
where CTOT represents the total investment cost (EUR), FC is the annual cash flow (EUR/yr), kdiscount and infl indicate the discount (%) and inflation rate (%), respectively assumed, while y represents the considered year.

4. Results

The implementation of the developed model within the DSS tool was aimed at helping the stakeholders in the decision-making process through a conscious comparison between different WEEE treatment and recovery scenarios. The development of a database of inputs to the model represented one of the project’s preliminary activities.
In Table 1, the parameters considered in the modeling study are summarized. They consisted of total WEEE amounts to be treated, specific WEEE composition (in terms of single categories), conferral costs for the different WEEE fractions, and market compensations (subsidies) for the recovered materials. The latter values were set according to the actual Italian market and also forecasting a possible future market evolution, according to interviews with specialized operators in the WEEE sector.
The annual report of the Italian WEEE Coordination Center, in fact, reports for 2022 a total WEEE production of 360,842 ton, subdivided into 5 categories (R1–R5): R1 includes refrigerating devices, and can be assimilated to fridges, while R2 mostly gathers washing machines and other large whites. R3 is related to monitors, including televisions, while R4 and R5 respectively include ICT products (including PCs) and light sources [39]. The relative proportion of the 5 WEEE categories are shown in Figure 4. The composition of the collected WEEE in Italy in 2022 highlights a prevalence of R2 (washing machines) and, secondly, of R1 (fridges) [39], coherently with what reported in Table 1. In Table 2, the most important output parameters considered in the assessment are summarized, including the technical constraints that were introduced to ensure plant economic and environmental sustainability. Besides traditional economic parameters, such as total capital cost, PB time, and NPV, meaningful environmental aspects were considered, including the reduction of CO2 emissions (due to resource recovery from WEEE) and the fraction of total recycled materials, expressed as % of the influent waste flow, to satisfy the thresholds of the current legislative framework (EU Directive 2012/19), that asks for a minimum 80% recovery.

Case Study: Friuli-Venezia Giulia WEEE Plant

The amount of WEEE generated in the Friuli-Venezia Giulia region (Table 3), which consists of a total production of about 7700 ton/yr, was used as the first attempt flux of input material for the plant modeling in the mathematical simulations. The different EWC (European Waste Catalogue) categories of the different WEEE products are reported in Table 3: it can be seen that specific WEEE production is around 6.5 kg/inhab yr, which is coherent with the mean North Italian production, and higher than mean Italian WEEE generation [39]. In addition, the WEEE composition used for the developed model is consistent with the most recent data regarding Italian WEEE production [39].
The main results of the mathematical modeling are depicted in Figure 5: the economic indices for a plant exclusively treating the local WEEE are generally not compatible with an industrial utility. In fact, the PB time is around 10 yr even for the most optimistic market scenarios, which forecast high conferral costs of the WEEE fractions and high revenues from the recovered secondary materials. The capital cost for the WEEE plant was calculated as EUR 7 M for the lowest amount of treated WEEE (7600 ton/yr), while it increased up to around EUR 30 M for the highest considered flux of WEEE (40,000 ton/yr).
The NPV index strongly depended on the economic market conditions and was essentially zero for the worst scenarios (corresponding to a PB time of 10 yr), while it increased up to EUR 30 M in correspondence to high waste conferral costs, high fees for recovered materials, and higher waste fluxes than those exclusively produced in the analyzed region (i.e., most favorable conditions). The recovered material fraction was in the range of 78–83%, and interestingly, an increase in the washing machine fraction allowed to stabilize the overall recovered fraction at about 80%, allowing to satisfy current legislation requirements. This is due to the relatively high amounts of materials that can be recovered from this particular WEEE category. An increase in plant potentiality up to 40,000 ton/yr (considering also treating the WEEE fluxes produced by other Italian regions) allowed for significantly improving the economic indices, reducing the PB time to about 4 yr for the most optimistic scenarios. In these conditions, a higher NPV value, up to EUR 30 M, was obtained. Regarding environmental performances, CO2 emission reduction was in the range of 8000–38,000 t CO2/yr, linearly depending on plant potentiality, due to the proportionally higher amount of secondary materials recovered from the treated WEEE.
For the considered WEEE treatment scenarios, the charts in Figure 6 depict the main effects of the most important decision variables on PB minimization and material recovery maximization. Some decision variables have been omitted in these charts due to the limited effect on the objective functions. In these diagrams, a box plot of a low level and a high level (represented respectively with a “−” and a “+”) for each variable is represented. The difference between the average value of the two groups represents the relative importance of the variable: the greater the difference, the more important the variable. In addition, the slope of the line connecting the mean values of the two groups represents the correlation between the variable and the objective. A positive slope indicates a positive correlation, while, on the contrary, a negative slope indicates a negative correlation.
The analysis of economic indices variation with the most relevant input parameters of the model (Figure 6) showed that an increase in washing machines and PC fractions could allow for reducing the investment PB time due to their relatively high conferral fees compared to other WEEE fractions (e.g., televisions) (Table 1). This behavior may be consistent with market evolution in the next years, considering that personal computers are more and more present in all household and industrial activities, and their life-span is often lower than other EEE products [6]. In addition, an increase in the fridge fraction and a decrease in PC percentage would allow to increase the total amount of recovered materials from the analyzed WEEE, according to the specific composition of each waste stream. In fact, the peculiar PC composition limits the amounts of recoverable materials, so it appears fundamental to treat a WEEE mix with a significant presence of other products, such as fridges. Finally, in the considered range of input variables (Table 1), PC conferral cost had a higher influence on plant economic sustainability, if compared to fridge conferral cost and also to the subsidies provided for secondary recovered material (steel and aluminum), due to the significantly higher boundary in the PC conferral cost that was considered in the assessment.
Actually, a significant increase in the subsidies provided by the Italian national consortium CONAI for recovered materials has been observed in the market in the latest years, thanks to the higher quality of the collected waste streams [40], so a further enhancement in these subsidies can be easily forecast in the near future. In any case, it is fundamental to monitor market conditions to assess the long-term sustainability of the investment. The presence of dedicated incentives could increase the financial sustainability of the WEEE treatment plant, especially when selecting a relatively low amount of treated waste, such as that exclusively produced in the analyzed region. This solution would create a real circular economy in the sector, but the calculated economic indices appear dissatisfying in the absence of dedicated incentives. Regarding environmental performances, CO2 emission reduction is mainly influenced by recovered material fractions: waste income showed to greatly influence environmental performances. The washing machine fraction displayed a similar but less pronounced effect, while the fridge fraction exhibited a limited effect. On the contrary, an opposite effect was featured by the PC fraction, leading to a decrease in CO2 emission reduction, again due to the limited possibility of material recovery due to its peculiar composition.

5. Discussion

The proposed DSS approach to assess WEEE plant sustainability and the related potential for resource recovery is extremely flexible and could be exported to different locations by changing boundary conditions (amount of treated WEEE, composition of the treated waste, conferral fees and subsidies for the recovered materials). This simplified evaluation, which accounts for multiple but potentially conflicting objectives (e.g., maximization of revenues and limitation of the environmental impacts), can be useful both for public authorities and private companies to preliminarily evaluate plant installation feasibility in changeable market conditions, including meaningful environmental aspects, which are becoming crucial in the transition towards a circular economy. Furthermore, additional treatment operations could be included in the modeled plant, such as the incineration of the non-recoverable fraction (mixed plastics, wood). The actual study could be a useful tool for policymakers and could be further implemented by considering the recovery potential of precious and rare earth metals, as proposed in [13] for the Australian case study, better assessing the overall value of the recovered materials. Furthermore, the inclusion of social indicators, such as additional job places, workdays created, and harm risk at work should be considered in the future to have a broader vision that goes beyond economic and environmental aspects; these indicators were studied in [34] in a WEEE multi-objective optimization model, together with conventional economic and environmental parameters.
From the preliminary results obtained in the present modeling campaign, it could be seen that installing a WEEE treatment plant in the analyzed region would be environmentally attractive considering a circular economy approach, and aims at closing the waste cycle in the area where it is produced, avoiding transporting the produced WEEE to other regions. In fact, plant implementation allowed a virtuous GHG emission reduction, comparable with what was obtained in similar studies [41]. If considering an economic perspective instead, WEEE plant sustainability would be assured only with a higher WEEE flow than that produced in the investigated region. In this regard, an enhancement of economic performances could be achieved through precious metals recovery (e.g., gold) from printed circuit boards (PCBs) of the treated WEEE as investigated by several studies [42,43]. However, WEEE plant installation may be seen also from the perspective of public utility, and not only from that of a private company, providing dedicated incentives to allow its operations. More generally, an effort towards a higher collection of WEEE is needed in Italy, due to the lower recovered amounts in comparison to other European countries, such as Germany and France [39], to move towards the required EU legislative standards. So, the amount of WEEE to be treated and recycled is expected to grow in the next years, even if with a slowly increasing trend, as observed in the past. The need for improved WEEE collection and management was underlined also in [32], where the Belgium case study was analyzed, reporting a low current recycling ratio of 32%; interestingly, the reported paper proposed an approach based on combined material flow analysis (MFA) and LCA to thoroughly optimize the environmental performances of the WEEE recycling chain.
Another remarkable paper suggested the possibility of applying the Internet of Things (IoT) approach in WEEE collection systems in Malaysia to enhance the efficiency of the collecting chain: a backend server was developed to automatically notify and schedule e-waste collectors to recover WEEE when the volume of the collector box reaches 80% filling [44]. As domestic e-waste flow is very complex and strongly depends on consumers’ behavior, household surveys can be conducted to obtain primary data to understand citizens’ behavior towards WEEE, including meaningful aspects such as out-of-use storage, intention to repair damaged EEE, waste destination, place and time of acquisition, second-time acquisition, and donation [45]. Other virtuous approaches to moving towards a circular economy in the WEEE sector may include a stronger focus on reuse, remanufacturing, and refurbishment as sustainable practices to extend product life, reducing the overall amount of produced waste [6].
Further development of the current model is needed in the future, since a deeper characterization of WEEE products and material composition should be carried out, following the fast WEEE market evolution that is occurring nowadays. In this framework, in [46], an intense characterization campaign was carried out to assess specific WEEE characteristics in Portugal: it was shown that the additives used to darken plastics lower their detection in installed optic separators, contributing to a lower global recycling ratio. Moreover, a more detailed mathematical model should be developed in the future, considering the energy recovery of the non-recoverable waste fractions that possess a significant energy recovery potential (e.g., wood or mixed plastics). Finally, advanced recovery technologies may be considered, such as biotechnological processes (e.g., bioleaching, biosorption, and bio-electrochemical systems), that represent new alternatives to traditional pyrometallurgical processes and that could be potentially exploited in the future to recover critical components and reduce environmental pollution [47]. However, the technological readiness level (TRL) and the environmental sustainability of these innovative strategies, which also include dissolution/precipitation, supercritical fluid extraction, catalytic pyrolysis, and waste-to-energy equipped with carbon capture and storage (CCS), should be carefully assessed before industrial implementation [48].
Finally, a dedicated tool for sustainability assessment in WEEE management named “SUSTWEEE” was recently developed in the scientific literature, addressing environmental, economic, and social aspects: seven main indicators, including qualitative and quantitative aspects, were defined and assessed considering 44 criteria [49]. Allocation and normalization steps were performed, resulting in a useful WEEE evaluation matrix with color coding; in this sense, it could be interesting to include social indicators in the model developed in the present study to give a broader overview to decision makers.

6. Conclusions

In this work, a mathematical toolbox was developed to preliminarily analyze the financial and environmental sustainability of a WEEE treatment plant with resource recovery, following circular economy principles. Different WEEE products were supposed to be treated coherently with the current market situation. The variability of relevant input parameters such as waste conferral fees, plant potentiality, percentage of the different WEEE fractions, and fees for the recovered material fractions were investigated. In order to assist stakeholders in the decision-making process, a holistic DSS was implemented, which can be potentially exported to other case studies across the world, considering the rising interest in WEEE management and the still limited recycling ratio, despite the recent legislation acts (e.g., UE Directive 2012/19).
The results revealed that the capital cost of WEEE plant construction is in the range of EUR 7–35 M for the different tested potentialities (8000–40,000 ton/yr), the lower bound corresponding to the WEEE exclusively generated in the region, while the upper bound considered treating also the WEEE produced by neighboring regions. The minimum obtained PB time was 4.3 yr, considering favorable market conditions (high conferral fees and maximum subsidies for the recovered product fractions) which can be consistent with future market evolution, considering the rising value of recovered materials, and the largest plant potentiality. CO2 emissions reduction due to material recovery was in the range of 8000–38,000 ton CO2/yr, linearly increasing with plant potentiality. The treatment of the WEEE exclusively produced in the investigated region allows to give significant environmental benefits, due to the establishment of a local circular economy; however, the financial sustainability of this solution is limited. Thus, proper financial incentives should be provided by public authorities. The fraction of recovered materials varied with WEEE- composition in the range of 78–83%, satisfying in most cases, the required threshold of 80% recovery. The specific composition of a WEEE mixture has a conflicting influence on results: while an increase in the PC fraction allows a rise in economic income, due to the high conferral fee, it decreases the fraction of recovered materials, due to the relatively unfavorable composition of this stream when compared to other fractions (e.g., washing machines).
Considering the rising amounts of WEEE produced globally and the currently low recycling rates, it will be critical in the future to develop a diffuse network of recycling plants. This research allows to preliminarily assess, in any situation, the financial and environmental sustainability of installing a WEEE treatment plant by specifying the boundary conditions (plant potentiality, WEEE composition, and conferral fees and subsidies for the recovered fractions). Further research and model optimization must be carried on in the future, such as the integration of energy recovery from non-recoverable fractions (mixed plastics and wood), as well as the consideration of social aspects, besides the economic and environmental parameters.

Author Contributions

Conceptualization, M.C., M.M. and P.S.; methodology, M.C. and M.M.; software, M.C.; validation, M.C. and M.M.; formal analysis, M.C. and M.M.; investigation, M.C. and M.M.; resources, P.S.; data curation, M.C. and M.M.; writing—original draft preparation, M.C.; writing—review and editing, M.M.; visualization, M.C., M.M. and P.S.; supervision, P.S.; project administration, P.S.; funding acquisition, P.S. All authors have read and agreed to the published version of the manuscript.

Funding

Research was funded by A2A, grant number CV19_005 DPIA UNIUD, CUP G66C19000000007.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data will be made available on request.

Acknowledgments

Matia Mainardis acknowledges CAFC S.p.A. for the collaboration in research activities.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Shekdar, A.V. Sustainable solid waste management: An integrated approach for Asian countries. Waste Manag. 2009, 29, 1438–1448. [Google Scholar] [CrossRef] [PubMed]
  2. Tonini, D.; Martinez-Sanchez, V.; Astrup, T.F. Material Resources, Energy, and Nutrient Recovery from Waste: Are Waste Refineries the Solution for the Future? Environ. Sci. Technol. 2013, 47, 8962–8969. [Google Scholar] [CrossRef] [PubMed]
  3. European Commission (EC). Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions. Closing the Loop—An EU Action Plan for the Circular Economy 2015; European Commission (EC): Brussels, Belgium, 2015. [Google Scholar]
  4. Kyriakopoulos, G.; Kapsalis, V.; Aravossis, K.; Zamparas, M.; Mitsikas, A. Evaluating Circular Economy under a Multi-Parametric Approach: A Technological Review. Sustainability 2019, 11, 6139. [Google Scholar] [CrossRef]
  5. Geissdoerfer, M.; Savaget, P.; Bocken, N.M.P.; Hultink, E.J. The Circular Economy—A new sustainability paradigm? J. Clean. Prod. 2017, 143, 757–768. [Google Scholar] [CrossRef]
  6. Shams, H.; Molla, A.H.; Ab Rahman, M.N.; Hishamuddin, H.; Harun, Z.; Kumar, N.M. Exploring Industry-Specific Research Themes on E-Waste: A Literature Review. Sustainability 2023, 15, 12244. [Google Scholar] [CrossRef]
  7. Favot, M.; Grassetti, L. E-waste collection in Italy: Results from an exploratory analysis. Waste Manag. 2017, 67, 222–231. [Google Scholar] [CrossRef] [PubMed]
  8. Román, E. 22—WEEE management in Europe: Learning from best practice. In Waste Electrical and Electronic Equipment (WEEE) Handbook; Goodship, V., Stevels, A., Eds.; Woodhead Publishing: Cambridge, UK, 2012; pp. 493–525. [Google Scholar] [CrossRef]
  9. Gollakota, A.R.K.; Gautam, S.; Shu, C.-M. Inconsistencies of e-waste management in developing nations—Facts and plausible solutions. J. Environ. Manag. 2020, 261, 110234. [Google Scholar] [CrossRef]
  10. Forti, V.; Balde, C.P.; Kuehr, R.; Bel, G. The Global E-Waste Monitor 2020: Quantities, Flows and the Circular Economy Potential; United Nations University/United Nations Institute for Training and Research: Bonn, Germany; International Telecommunication Union: Geneva, Switzerland; International Solid Waste Association: Rotterdam, The Netherlands, 2020. [Google Scholar]
  11. Shittu, O.S.; Williams, I.D.; Shaw, P.J. Global E-waste management: Can WEEE make a difference? A review of e-waste trends, legislation, contemporary issues and future challenges. Waste Manag. 2021, 120, 549–563. [Google Scholar] [CrossRef]
  12. Sahajwalla, V.; Gaikwad, V. The present and future of e-waste plastics recycling. Curr. Opin. Green Sustain. Chem. 2018, 13, 102–107. [Google Scholar] [CrossRef]
  13. Islam, M.T.; Huda, N. Assessing the recycling potential of “unregulated” e-waste in Australia. Resour. Conserv. Recycl. 2020, 152, 104526. [Google Scholar] [CrossRef]
  14. Castro, F.D.; Bassin, J.P. Electronic waste: Environmental risks and opportunities. In Hazardous Waste Management; Elsevier: Amsterdam, The Netherlands, 2022; pp. 421–458. [Google Scholar] [CrossRef]
  15. Perkins, D.N.; Drisse, M.-N.B.; Nxele, T.; Sly, P.D. E-Waste: A Global Hazard. Ann. Glob. Health 2014, 80, 286–295. [Google Scholar] [CrossRef] [PubMed]
  16. Zuo, L.; Wang, C.; Sun, Q. Sustaining WEEE collection business in China: The case of online to offline (O2O) development strategies. Waste Manag. 2020, 101, 222–230. [Google Scholar] [CrossRef] [PubMed]
  17. Andersen, T.; Jæger, B.; Mishra, A. Circularity in Waste Electrical and Electronic Equipment (WEEE) Directive. Comparison of a Manufacturer’s Danish and Norwegian Operations. Sustainability 2020, 12, 5236. [Google Scholar] [CrossRef]
  18. European Commission (EC). Directive 2012/19/EU of the European Parliament and of the Council of 4 July 2012 on Waste Electrical and Electronic Equipment (WEEE) 2012, 24–38. Available online: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32012L0019 (accessed on 15 September 2023).
  19. Wäger, P.A.; Hischier, R.; Eugster, M. Environmental impacts of the Swiss collection and recovery systems for Waste Electrical and Electronic Equipment (WEEE): A follow-up. Sci. Total Environ. 2011, 409, 1746–1756. [Google Scholar] [CrossRef]
  20. Tansel, B. From electronic consumer products to e-wastes: Global outlook, waste quantities, recycling challenges. Environ. Int. 2017, 98, 35–45. [Google Scholar] [CrossRef]
  21. Yin, J.; Gao, Y.; Xu, H. Survey and analysis of consumers’ behaviour of waste mobile phone recycling in China. J. Clean. Prod. 2014, 65, 517–525. [Google Scholar] [CrossRef]
  22. Ongondo, F.O.; Williams, I.D.; Cherrett, T.J. How are WEEE doing? A global review of the management of electrical and electronic wastes. Waste Manag. 2011, 31, 714–730. [Google Scholar] [CrossRef]
  23. Thapa, K.; Vermeulen, W.J.V.; Deutz, P.; Olayide, O. Ultimate producer responsibility for e-waste management—A proposal for just transition in the circular economy based on the case of used European electronic equipment exported to Nigeria. Bus. Strategy Dev. 2023, 6, 33–52. [Google Scholar] [CrossRef]
  24. Kummer, S.; Löhle, S.; Schmiedel, U. Consumer survey on the final consumer behavior concerning the disposal of WEEE in Germany. Waste Manag. Res. J. Sustain. Circ. Econ. 2022, 40, 538–544. [Google Scholar] [CrossRef]
  25. Menikpura, S.N.M.; Santo, A.; Hotta, Y. Assessing the climate co-benefits from Waste Electrical and Electronic Equipment (WEEE) recycling in Japan. J. Clean. Prod. 2014, 74, 183–190. [Google Scholar] [CrossRef]
  26. Banaszkiewicz, K.; Pasiecznik, I.; Cieżak, W.; Boer, E.D. Household E-Waste Management: A Case Study of Wroclaw, Poland. Sustainability 2022, 14, 11753. [Google Scholar] [CrossRef]
  27. Pérez-Martínez, M.M.; Carrillo, C.; Rodeiro-Iglesias, J.; Soto, B. Life cycle assessment of repurposed waste electric and electronic equipment in comparison with original equipment. Sustain. Prod. Consum. 2021, 27, 1637–1649. [Google Scholar] [CrossRef]
  28. Boldoczki, S.; Thorenz, A.; Tuma, A. The environmental impacts of preparation for reuse: A case study of WEEE reuse in Germany. J. Clean. Prod. 2020, 252, 119736. [Google Scholar] [CrossRef]
  29. Su, J.; Li, C.; Tsai, S.-B.; Lu, H.; Liu, A.; Chen, Q. A Sustainable Closed-Loop Supply Chain Decision Mechanism in the Electronic Sector. Sustainability 2018, 10, 1295. [Google Scholar] [CrossRef]
  30. Cottes, M.; Mainardis, M.; Goi, D.; Simeoni, P. Demand-Response Application in Wastewater Treatment Plants Using Compressed Air Storage System: A Modelling Approach. Energies 2020, 13, 4780. [Google Scholar] [CrossRef]
  31. Isernia, R.; Passaro, R.; Quinto, I.; Thomas, A. The Reverse Supply Chain of the E-Waste Management Processes in a Circular Economy Framework: Evidence from Italy. Sustainability 2019, 11, 2430. [Google Scholar] [CrossRef]
  32. De Meester, S.; Nachtergaele, P.; Debaveye, S.; Vos, P.; Dewulf, J. Using material flow analysis and life cycle assessment in decision support: A case study on WEEE valorization in Belgium. Resour. Conserv. Recycl. 2019, 142, 1–9. [Google Scholar] [CrossRef]
  33. Jian, J.; Guo, Y.; Jiang, L.; An, Y.; Su, J. A Multi-Objective Optimization Model for Green Supply Chain Considering Environmental Benefits. Sustainability 2019, 11, 5911. [Google Scholar] [CrossRef]
  34. Dutta, P.; Mishra, A.; Khandelwal, S.; Katthawala, I. A multiobjective optimization model for sustainable reverse logistics in Indian E-commerce market. J. Clean. Prod. 2020, 249, 119348. [Google Scholar] [CrossRef]
  35. ARPA (Environmental Protection Regional Agency) Friuli-Venezia Giulia. Waste Production Database 2022. Available online: http://www.arpa.fvg.it/cms/tema/rifiuti/ (accessed on 2 July 2023).
  36. Pohekar, S.D.; Ramachandran, M. Application of multi-criteria decision making to sustainable energy planning—A review. Renew. Sustain. Energy Rev. 2004, 8, 365–381. [Google Scholar] [CrossRef]
  37. Simeoni, P.; Ciotti, G.; Cottes, M.; Meneghetti, A. Integrating industrial waste heat recovery into sustainable smart energy systems. Energy 2019, 175, 941–951. [Google Scholar] [CrossRef]
  38. Bejan, A.; Tsatsaronis, G.; Moran, M.J. Thermal Design and Optimization, 1st ed.; John Wiley & Sons: New York, NY, USA, 1996. [Google Scholar]
  39. Italian WEEE Coordination Centre. WEEE Collection Yearly Report; Italian WEEE Coordination Centre: Genova, Italy, 2022.
  40. CONAI. Subsidies for Recovered Materials Provided by the National Italian Consortium for Packaging 2023; CONAI: Rome, Italy, 2023. [Google Scholar]
  41. Park, J.; Jung, I.; Choi, W.; Choi, S.O.; Han, S.W. Greenhouse gas emission offsetting by refrigerant recovery from WEEE: A case study on a WEEE recycling plant in Korea. Resour. Conserv. Recycl. 2019, 142, 167–176. [Google Scholar] [CrossRef]
  42. D’Adamo, I.; Ferella, F.; Gastaldi, M.; Maggiore, F.; Rosa, P.; Terzi, S. Towards sustainable recycling processes: Wasted printed circuit boards as a source of economic opportunities. Resour. Conserv. Recycl. 2019, 149, 455–467. [Google Scholar] [CrossRef]
  43. D’Adamo, I.; Rosa, P.; Terzi, S. Challenges in Waste Electrical and Electronic Equipment Management: A Profitability Assessment in Three European Countries. Sustainability 2016, 8, 633. [Google Scholar] [CrossRef]
  44. Kang, K.D.; Kang, H.; Ilankoon, I.M.S.K.; Chong, C.Y. Electronic waste collection systems using Internet of Things (IoT): Household electronic waste management in Malaysia. J. Clean. Prod. 2020, 252, 119801. [Google Scholar] [CrossRef]
  45. Rodrigues, A.C.; Boscov, M.E.G.; Günther, W.M.R. Domestic flow of e-waste in São Paulo, Brazil: Characterization to support public policies. Waste Manag. 2020, 102, 474–485. [Google Scholar] [CrossRef] [PubMed]
  46. Martinho, G.; Pires, A.; Saraiva, L.; Ribeiro, R. Composition of plastics from waste electrical and electronic equipment (WEEE) by direct sampling. Waste Manag. 2012, 32, 1213–1217. [Google Scholar] [CrossRef] [PubMed]
  47. Işıldar, A.; Van Hullebusch, E.D.; Lenz, M.; Du Laing, G.; Marra, A.; Cesaro, A.; Panda, S.; Akcil, A.; Kucuker, M.A.; Kuchta, K. Biotechnological strategies for the recovery of valuable and critical raw materials from waste electrical and electronic equipment (WEEE)—A review. J. Hazard. Mater. 2019, 362, 467–481. [Google Scholar] [CrossRef]
  48. Arena, U.; Ardolino, F. Technical and environmental performances of alternative treatments for challenging plastics waste. Resour. Conserv. Recycl. 2022, 183, 106379. [Google Scholar] [CrossRef]
  49. Gavrilescu, D.; Enache, A.; Ibănescu, D.; Teodosiu, C.; Fiore, S. Sustainability assessment of waste electric and electronic equipment management systems: Development and validation of the SUSTWEEE methodology. J. Clean. Prod. 2021, 306, 127214. [Google Scholar] [CrossRef]
Figure 1. Conceptual scheme of the developed decision support system (DSS).
Figure 1. Conceptual scheme of the developed decision support system (DSS).
Sustainability 15 16248 g001
Figure 2. Recoverable and non-recoverable fractions from the analyzed WEEE streams.
Figure 2. Recoverable and non-recoverable fractions from the analyzed WEEE streams.
Sustainability 15 16248 g002
Figure 3. Process scheme of the waste electric and electronic equipment treatment plant.
Figure 3. Process scheme of the waste electric and electronic equipment treatment plant.
Sustainability 15 16248 g003
Figure 4. Composition of the collected waste electric and electronic equipment in Italy in 2022.
Figure 4. Composition of the collected waste electric and electronic equipment in Italy in 2022.
Sustainability 15 16248 g004
Figure 5. Variation of WEEE plant environmental and economic indices at different plant potentialities.
Figure 5. Variation of WEEE plant environmental and economic indices at different plant potentialities.
Sustainability 15 16248 g005
Figure 6. Effect of waste flux, composition and conferral fees on waste electric and electronic equipment plant payback time, CO2 emission reduction, and recovered material fraction (PC: personal computer; WM: washing machine).
Figure 6. Effect of waste flux, composition and conferral fees on waste electric and electronic equipment plant payback time, CO2 emission reduction, and recovered material fraction (PC: personal computer; WM: washing machine).
Sustainability 15 16248 g006
Table 1. Variable input parameters considered in the modeling approach.
Table 1. Variable input parameters considered in the modeling approach.
ParameterInitial ValueSimulated RangeSimulation Step
WEEE plant potentiality (ton/yr)76007600–40,0005000
Fridges specific fraction (-)0.170.15–0.200.01
Washing machines specific fraction (-)0.290.25–0.400.01
Computers specific fraction (-)0.140.10–0.250.01
Fridges and washing machines conferral cost (EUR/ton)8050–20010
Televisions conferral cost (EUR/ton)10050–12010
Computers conferral cost (EUR/ton)400200–50020
Compensation steel recovery (EUR/ton)3131–6110
Compensation aluminum recovery (EUR/ton)131131–25120
Compensation plastic recovery (EUR/ton)8282–40440
Compensation glass recovery (EUR/ton) 66–539
Table 2. Output parameters considered in the modeling approach and adopted constraints.
Table 2. Output parameters considered in the modeling approach and adopted constraints.
ParameterConstraint
Total investment cost (EUR)None
Yearly operating costs (EUR/yr)None
Yearly income (EUR/yr)None
Net Present Value (NPV) (EUR)None
Pay-back (PB) time (yr)≤10 yr
CO2 emission reduction (ton CO2/yr)None
Recovered material fraction (%)≥80%
Table 3. WEEE production in Friuli-Venezia Giulia region in the year 2022, sub-divided using EWC codes.
Table 3. WEEE production in Friuli-Venezia Giulia region in the year 2022, sub-divided using EWC codes.
ProvinceEWC 200307 (ton/yr)EWC 200123 (ton/yr)EWC 200135 (ton/yr)EWC 200136 (ton/yr)EWC 200121 (ton/yr)WEEE tot (ton/yr)Popul (inhab)Specif Prod (kg/inhab yr)
Udine-8685892057223363516,7156.51
Gorizia-1811135783978137,7847.10
Pordenone824192451129111798309,6125.80
Trieste-32217091631501228,0806.58
Total Friuli-Venezia Giulia region821790111746803977081,192,1916.47
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Cottes, M.; Mainardis, M.; Simeoni, P. Assessing the Techno-Economic Feasibility of Waste Electric and Electronic Equipment Treatment Plant: A Multi-Decisional Modeling Approach. Sustainability 2023, 15, 16248. https://doi.org/10.3390/su152316248

AMA Style

Cottes M, Mainardis M, Simeoni P. Assessing the Techno-Economic Feasibility of Waste Electric and Electronic Equipment Treatment Plant: A Multi-Decisional Modeling Approach. Sustainability. 2023; 15(23):16248. https://doi.org/10.3390/su152316248

Chicago/Turabian Style

Cottes, Mattia, Matia Mainardis, and Patrizia Simeoni. 2023. "Assessing the Techno-Economic Feasibility of Waste Electric and Electronic Equipment Treatment Plant: A Multi-Decisional Modeling Approach" Sustainability 15, no. 23: 16248. https://doi.org/10.3390/su152316248

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop